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A 3 S T R ACT 



This report explores the effectiveness of bilingual education based 
on a study or the literature. The study ^s undertaken as ^alt or III 
ounTitL ry /T etf ^°«^ge-Minority Notice of Proposed Rulemaking 

If HJS^T J" 0 in re3pOQ3e 50 TiCle VI °* che Civil Rights Act 

1 M L h0 " gh che cur " nc Administration has withdrawn the proposed 

£ "ta £u M 5 examination of che apartment of Education's policy 

on whe education or language-minority children continues. 

chj .r S!f S "r 1 / 3 - 6 !??^ 3117 cricical in ^ght of this review's findings 
a^™5? ffleCh0 l 1 0 f bi ^ n gual instruction (transitional bilingual education 

Affairs' ST^nS? ° f f " ° f BiUnSUal Educacion and Minority ^ge 
™L h! 7 Ch L 0ffice for Righ" in che Department of Education 

2£Se£? 3UPPOrCed 33 Che 3012 °« chod ""d with language-minority 

which'^^^f 1 W ! U Wet 300 docunen « concerning bilingual education- 
ChlS °°" con P" he «ive review to date on this subject. 
f US ? °° 3C ° £ Che bilin g ual "udies, especially Title VTI 
lZ^ Zi ° a ^ ° f S°° r qU3liC7 ' our "nclusions^have been 
5 oJrlonl^? 0nl7 28 3CUd±eS ChaC We " -'^logically applicable 

Our conclusions, briefly suumarized, are these: 

o Schools can improve the achievement level of language-minority 
children through special programs. 7 

o The case for the effectiveness of transitional bilingual education 
is so weak that exclusive reliance on this instruction method is 
clearly not justified. Too little is known about the problems of 
STJii 08 , Un?Ua ? e prides co prescribe a specific remedy at 
«ch Jf ; Therefore ' ««iag civil rights guarantees, 

each school district should decide what type of soecial program is 
most appropriate for its own unique setting. 

o There is no justification for assuming that it is necessary to 
ceach nonlanguage subjects in the child's native tongue in order 
for the language-minority child to make satisfactory progress in 
school However, if nonlanguage subjects are to be caught in 
anglish, che curriculum muse be scruccured diff erectly from che 
^udenL?^^ 11111111 13 3Cnictured for ^lingual Snglish-speakir.g 

o Immersion programs, which involve scruccured curriculums in 
English for both language and nonlanguage subjecc areas, show 
promising resulcs and should be given O ore accencion in program 
developmenc. ^ 8 



The Tide VTI program for bilingual educacion muse cake seeps 
co improve che qualicy of its program evaluations. 



■ERIC 



O 

O 



SUMMARY REPORT OF A REVIEW OF THE LITERATURE ON THE 
EFFECTIVENESS OF BILINGUAL EDUCATION * 

Introduction 

This investigation* was begun at the request of the White House 
Regulatory Analysis and Review Group for an assessment of the effectiveness 
of transitional bilingual education (TBE). The request cane during that 
group s review of the Department of Education's proposed language-minority 
regulations which were issued in August 1980 in response to Title VI of the 
Civil Rights Act of 1964. Although the proposed rules have been withdrawn, 
the question or the effectivenss of transitional bilingual education is. 
still important for the development of Department policy. 

The review focused on cwo questions derived from the principal intent of 
rederal policy: 

1. Does transitional bilingual education lead to better 
performance in English? 

2. Does transitional bilingual education lead to better 
performance in nonlanguage subject areas? 

Although a number of other goals are often recognized for bilingual education 
(e.g., reduced dropout rates, improved self-image and attitude toward school, 
preservation of the primary language and culture, and lower absenteeism), we 
limited this review to these two major questions. Few of. the studies reviewed 
addressed the other goals, and a systematic assessment of their accomplish- 
ments could not be nade. 

This review did not directly include all the evaluations of bilingual 
programs that have been completed. The studies reviewed were subject to the 
following limitations: 

o In general, we did not review studies rejected as unsound by easier 
reviewers (Zappert and Cruz, 1977; Engle, 1975). An effort vas made 
to examine all studies reported since Zappert and Cruz, the most 
recent prior review. 

o The Office of 'Bilingual Education (Title VII program) was unable to 
provide copies of its pre-1978 evaluations, so most of them were noc 
available. However, since Zappert and Cruz (1977) seem to have re- 
viewed and rejected most of the pre-1978 Title VII evaluations, we 
believe the present review is the most comprehensive review of the 
effectiveness of bilingual education yet undertaken. 

Since our focus was on transitional bilingual education, our limited 
time and resources prohibited an equally comprehensive coverage of 
alternative methods. However, we have covered the major studies. 
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Consideration of Che literature and Federal policy led to the identifi- 
cation of three basic instructional alternatives, in addition to the alterna- 
tive of doing nothing for the language-minority child (also known is 
submersion) : 

o Submersion , Language-minority children ars placed into an ordinary 
classroom where English is spoken. There is no special program to 
help them overcome- the language problem. Submersion is. aptly de- 
scribed as -fink or swim," The minority home language (LI) is not 
used at all in the classroom, 3 

o Structured Immersion , Instruction is in the second language (L2), 
as in the case of submersion, but there are important differences, 
/The immersion teacher understands the home language (LI), and 
students can address the teacher in the home language (Li); the 
immersion teacher, however, replies only in the second language 
(L2), Furthermore, the curriculum is structured so that no prior 
knowledge of the second language (L2) is assumed when subject areas 
are taught. All cbg^d^vJtsCkxt^ 

stoo3TTjy^tfaC«udeKts, The students in effect learn L2 and content 
simultaneously,] Structured immersion differs from bilingual instruc- 
tion in that the home language (LI) is never spoken by the teacher 
and subject area instruction is given in the second language from 
the beginning, 

o English as a Second Language (ESL) , ESL students are placed in reg- 
ular' instruction for most of the day. During part of the day, how- 
ever, these students receive extra instruction in English, Generally, 
this extra help is based on a special curriculum designed to teach 
English as a second language. Home language (LI) may or may not be 
used in ESL instruction, 

o Transitional Bilingual Education (TBE) , Subject matter is taught 
in the home language (LI) until the students' second language (L2) 
(English) is good enough for them to participate successfully in a 
regular classroom. ESL is often used to help minimize the time needed 
to master English. Home language (LI) instruction is gradually 
phased out, and regular English instruction is gradually phased in. 
TBE is differentiated from submersion and ESL by the use of the home 
language (LI) for instruction. 

These three instructional types sometimes shade into one another; for 
example, most TBE programs include an ESL component. In addition v there is 
a considerable range of activities incorporated within each type. Neverthe- 
less, the typology is real and important. If the types are thought of as 
representing different philosophies for addressing the needs of students with 
limited English proficiency, it is immediately apparent that the different 
philosophies lead to very different classroom practices which can be identi- 
fied in actual classroom settings. 
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The alternative instructional models differ on both civil rights and 
educational dimensions ♦ Submersion is the absence of a special program which 
the Supreme Court found to* violate the civil rights of language-minority 
children in the case of Lau y. Nichols . Lau was a class-action suit against 
the San Francisco Public School District which alleged that the district's 
failure to provide special educational services to non-English-speaking 
Chinese students violated both the equal protection clause of the iAth Amend- 
ment and Title VI of the Civil Rights Act of 1964c The Supreme Court: found 
the San Francisco Public School District to be in violation of Title VI. 
However, the Court declined to prescribe a specific program that would pro- 
vide equal educational benefits, stating: "Teaching English to the students 
of Chinese ancestry is one choice. Giving instruction to this group in 
Chinese is another. There may be others Each of the three instructional 
methods seeks to correct the civil rights problem described by the Court in 
the Lau decision by providing special help to the language-minority child. 

The differences among the three methods can be further illustrated in a 
brief outline of the underlying arguments supporting each method as a success- 
ful solution for the problems of language-minority children: 

o Transitional Bilingual Education * While children are learning 
English, they should be taught subject material in their home lan- 
guage so that their academic progress will not be retarded by their 
limited knowledge of English. It is easier to first learn to read 
in the home language than in the second language and the reading in 
the home language will facilitate second-language reading. There- 
fore, the sequence of instruction in LI before L2 is superior to an 
all-second-language program. 

o English as a Second Language . Concentrated additional instruction in 
English language skills will keep the students from falling behind in 
subject areas. 

o Structured Immersion . The solution to developing English and pro- 
gressing in other subjects simultaneously is to teach all subjects 
in English at a level understood by the students. Although the 
curriculum assumes no prior knowledge of English language-minority 
students in effect learn English as they learn math, and learn aiach 
through English instruction that is understandable at their level 
of English proficiency. 



Methodological Approach 

In reviewing a body of research to determine the effectiveness of a par- 
ticular instructional program, three fundamental questions are asked: 

1. Docs the study present data relevant to Che issues of interest? 

2. Does the design of the study permit any plausible alternative 
explanation for the results other than th?t the program worked? 

3. How widely can the results of acceptable studies be generalized? 

3 
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•.eJSiST!" JTt c Zh±3 revi6W iS Ksed on che ^Plication of scandara 

S ^ C 2 C ? f ° r Che adequac> Jf research de3i ^ which are^idSy 
«Sf! Jf «»«cacion liceracure. These criteria ara\pplied co che 

iSfi; S ^ " 3 " " Ch6y 41:8 of »cctpc«bl. quality/ S chis 

rf^; , 6 w CriCet J a f ° r ™ ch °dological soundness were applied in 7way chac 
atSSJh ? ^ f dMi8n aMkMM ia on « a « a «n be comSensaled for by 

eS%731 2/2Rf ' PPCrt a f CrU2 (1977) re ^ ecced che « ud ? 
aSScy ^ i0g C % conC y o1 f « differences in language 

create Si ^JS'* 1 " 'T 7 * 3CUdy randoaly assi 8 ned « ud encf <° 

5^.&rS?.22«£ c ™ ex?eriaenc) * no further concro1 of 

uacio^cuSes!"^^ '""V* and 

p ffnHM *- ^poexx ana Stanley (1963), tor example, Use 12 broad cat- 

£S2S KLS* £.12=2 r^-T£^'it 

rhZ f 8Vi6W " Ch SCudy was assessed codecermine if it addressed 

o The scudy did noc address our issues . 

° If ""denes were noc randomly assign co che creacmenc and 

i252 S S?ff 0UP3 ' + 3 v d n ° Ching done co concro1 for possible 
^ Jfif an - M beWeen the Sr °^ 3 ' Che acudies-were rejecced. 

Any differences round becveen che scudencs in che special program 

co B reexS c °in P . n H?ff in SpeCial pr ° 8ran could have been due 

, Pr "f" iDg differences oecween che cvo groups.' Differences need 

v*re^ e ec% e d n J ° f ^ tae ° f Che ^ <** ^ 

° m J f udla f,- d ° 0C apply a PP r °P^ce scacigcical cescg co demonscrace 
program ef feces, che scudies were noc accepced. Presencing differences 
between two groups is noc sufficienc proof chat che differences "id 
JnL r£%2 r „ ChaaCe ' Theref0 "' scaciscical cescs muse be introduced 
Phe^o^on 3 ^ 7 ^ " ^ ^ we " n « a «*ance 

° 1lJ^ d l ChC ° ora - refe »°ced design .6 Some studies form what 

™Z S S * "u 11 " 0 - 1 group b7 con P«i««Vowch againsc che cesc 

VSZL ^ n * SCUdiM Ch6Ck " 366 if 3tudencs in che 3 P"i^ 
program showed a gain againsc che norm. In chis design, che race of 

progress of ch. bilingual child is compared with che £ e of progress 
of che monolingual norming groups. There is no reason co believe 
chac che race of progress of bilingual and monolingual scudencs is 
cue 3 aoe • 
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Therefore, any differences found by using a norm-referenced design 
cannot be attributed to the effects of the program. 

* 0 1118 examined gains over the school year without a con trol 

C2HE # Most students learn something over the school year, so" 
their scores will increase. If we want to know whether students 
gained more by being in the special language program than they would 
have gained in a regular school program, a comparison group of reg- 
ular school students must also be included. 

o The study used grade-equivalent scores . 3 Grade-equivalents do not 
correspond to the time pattern of learning, and the methods used to 
produce them are inaccurate. Equal grade-equivalent gains for two 
students may not represent equal learning. Use of grade-equivalents 
has often been criticized by evaluation experts. Quoting from "A 
Prototype Guide to Measuring Achievement Level and Program Impact on 
Achievement in Bilingual Projects" by Horst et al. (1980); 

Grade-equivalent scores provide* an illusion of simplicity but, 
in fact, they are almost impossible to interpret, aven for 
specialists in test construction. Grade-equivalent scores 
should never be used by anyone for any purpose whatsoever- 

However, there is disagreement among testing experts whether grade- 
equivalents are totally unacceptable for measuring student achieve- 
ment. One study that would have been accepted except for its use of 
grade-equivalents has been separately identified (Olesini, 1971), 

By the same token, acceptable studies were— 

o True experiments in which students were randomly assigned to treat- 
ment and control groups, or 

o Studies using nonrandom assignment which controlled for possible 
preexisting differences between the groups either by mauching stu- 
dents in the treatment and comparison groups or through using sta- 
tistical procedures. 

Analysis of covariauce was by far che nose common scaciscical aethod 
used co control for preaxisting differences which could influence achievement 
between groups. Many people have serious reservations about whether this 
method succeeds in properly adjusting preexisting differences. Similarly, 
there are doubts thai: matching is entirely successful. For this at>alvsis, 
we generally considered both methods to be acceptable unless there were 
defects in the application of these methods. 

There are two ways results can acquire generalizability. First, the 
students studied can be selected in sucn a way as to be representative of the 
entires population of students in whom we are interested — in this case 
language-minority children in the United States. Only 1 of the more Lan 
300 documents we reviewed comes close to acquiring national gene'ralizabilitv 
(Danoff et al., 1977, 1978) and this study has problems. Second, generaliz- 
ability comes from consistent findings in many different settings. Thus i-' 
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^li^ individual study, the weight of the collective evidence 

could -be ccapelling. Since only one of the studies we reviewed was nationally 

^tion wort! Wh6n arriVing 3C COnclu3ion3 as » »»* bilingual 



Results 



Of the several hundred, studies covered by the -view, only 28* were -"ound 
to apply to our concerns and to meet our methodological criteria 3ef"e ois- 
no« chat IT^f ~ f °T d " ta « cl »dologically acceptable, we SSS 
evident £ S^*"!* 1 3CUdieS ChaC hav « Previously been widely cited as 
evidence for the .effectiveness of TBE to be methodologically unacceptable 
SrSfTf* 0 ^,^ T T Oukomaa, 1976; Skucnabb-Kangas , 1979; 7 S u2w ley 

LlL^h ^ 13 u 77 l Wb ** 19785 Trevin °' 1968; Modiano, 1968; Egan and 7 ' 
Goldsmith, 1981; Rosier and Holm, 1980; and AIR, 1975a, 1975c, 1975e).9 

Table 1 summarizes the 28 studies we found co apply to our wo questions 

St?? ™ e ^ lMl u a,eCh0d0l0gical criCeria ' * comparison, Za^pert and Cruz 
(1977) found 18 methodologically acceptable studies! For each study cable 1 

S rl T T ' ? 8MdM ° f 3Ch ° 01 enc «P-«ed, the number oflLencs 
£1 \ J, rl mnZ f C °! Cr01 810,11)8 conbined . =he languages used by the pro- 
sSL Th^ r83U i CS ^ "P 0 "** for second-language and macE 

n0 " f " qUenC h0M ^"SW Spanish, but a number of other 
languages were represented as --ell. The most common second language "as 
English. In three studies, French was Che second language. Most of the 
studies were neither longitudinal nor true experiment!. leverS studies 
included very large numbers of students. s -ua3.es 

For each study we examined, cable 1 indicates whether the study was 
on 2L22 °LT iValenC C ° an ° Cher a PP" ac h. These comparisons were based 
resSS ,.r rB 3Caci3Cicall y significant. Some studies had mixed 

Sun * ^ ? ^" ° n " S " " 313(16 levels * i*™* " 3U 1« are 

found, we have indicated the nature of the different results. 

Structured immersion programs seem to have done particularly well. 10 
Lambert and Tucker (1972) and Barik and Swain (1975) Lund second-language 
£S ^jSPSS """^ ^»ion superior to ESL, and Pena-Hughes and 
Sut! Jill \ < d 3CrucCured lesion superior to. transitional bilingual 
education. As for nonlanguage subjects, Lamberc and Tucker (1972), Barik 
rLu u )f ^/f 003 8C 3l - C1967) all showed chat it is possible to 
teach math successfully in the second language. This finding suggests that 
if the curriculum is properly structured so that the means of communication 
is ac a level the child can understand, chere will be no negativTSnse- 
quences from teaching math in the second language. We found no data in 
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TABLE I 



SUMMARY OF APPLICAUUS S'lUOJES 



Author Date Grade 



Barik 
et al . 



1977 



Carsrud 
a ml 

Curtis 



1980 



AIR 1975b K-i 

(Corpua 
Chrlati) 

Ames and 1976 1-9 
Uicka 



Balasu- 1973 K-3 
braaon- " 
lan et al* 



Barlk and 1975 K-2 
Swain 



2-5 



4-5 



Design * 



Number ^ 
of 

Students 



I>angua».;ea - 
Li L2 



Longitudinal; 
analysis of 
covarlance 

Analysis of 
covarlance 



Analysis of 
covarlance and 
other adjust- 
ments 



393 Spanish English 



669 Spanish English 
and 

French 

317 Spanish English 



Longitudinal; 2,253 
analysis of 
covarlance 



Longitudinal; 
analysis of 
covarlance 



Longitudinal j 
analysis of 
covarlance 



English French 



English French 



172 Spanish English 



1.2 



Rep or ted Results 



Math 



TUE no different from submersion 
in 1 grade; TUB better than 
submersion in 1 grade 



THE no different from 
ESL alone 



TBE no different from 
ESL alone 



Iiwuersion better than 
ESL 



TBE better than submer- 
sion in 1 grade; TUE 
no different from 
submersion In 1 grade 



TUE better than 
ESL alone 



Math taught in 
L2 no different 
from math taught 
in U 

Depending on 
year and grade, 
math taught in 
L2 was worse 
than, no diff- 
erent from, or 
even better 
than math 
taught in LI 

TUE no different 
from submersion 



(Continued) 



TABLE 



Author 



Date Grade 



A. Cohen 1975 K-3 



Cottrell 1971 



K-l 



Design 



t 



I. SUMMARY OF APPLICABLE STUDIES (Continued) 



Number 2 
or' 

Students 



Longitudinal 
analysis of 
covarlauce 
am! other 
adjustments 



Analysis cjf 
covarlauce 



Languag e a ^ 
LI L2 



90 Spanish English 



470 Navajo English 



L2 

TBE no different from 
submersion on 86 of > 
100 language skills; 
submersion better than 
TBE on 11; TBE better 
than submersion on 3 



TBE no different 
from submersion 



Reported Results 



Math 

TBE no different 
from submersion 
lit 2 of 3 
grades; TBE 
better than sub- 
mersion in 1 
grade. 



Covey 



1973 



Danoffe 1977, 2-6 
et al. 1978 



lluzar 1973 2-3 



Kaufman 



1968 



Junior 
High 



Lambert 
ami 
Tucker 



1972 



1-4 



12 



Random ■ 
assignment 

Analysis of 
covarlance 
and other 
adjustments; 
big study 

Random assigns 
went; one-way 
analysis of 
covarlance 

Experiment; 
longitudinal 



longitudinal; 
analysis of 
"covarlance' 



200 Spanish English TBE better than sub- 

mersion 



8,900 Several English 



160 Spanish English 



139 Spanish English 



213 English French 



(Continued) 



Submersion better than 
TBE 



TBE no different from 
subme rsion 



TBE better than submersion 
on 2 component scores of 
a standardized achievement 
teat and no different on 7 
component scores in one 
school; TBE no different 
from submersion on 9 
tests in another school. 



TBE no different 
from submersion 

TBE no different 
from submersion 



Math taught in 
1/2 no di fferent 
from math 
taught in Li 

13 



Author Date Grade Desig n* 



Legar- 1979 
reta 



Luw 



1971 



Matthews 1979 2,4, 
6,8 

McConnell 1980 Pre-K 
-3 



McSpadden 1979 K-l 



McSpadden 1980 K-2 



Moore and 1978 K-2 
Parr 



TABLE K SUMMARY OF APPLICABLE STUDIES (Continued) 

Reported Keuulta 



Number ^ 
of 

Students 



Languages 3 
LI 1.2 



L2 



Analysis of 
covariance 



Random 
assignment 



Log-linear 
model 



80 



Spanish ting 11 ah 



55 Chinese English 



1,011 Many 



English 



Longitudinal; 1,020 Spanish English 
subject as 
own control 



Analysis of 
covariance 

Longitudinal ; 
analysis of 
cova rlance 



Analysis of 
covariance 



196 French English 
263 French English 



130 Spanish English 



THE better than submersion 
or THE no different from 
submersion, depending on 
the test; TBE with ESL better 
than TBE without ESL component 

ESL alone better than TOE on 
3 testa; ESL alone no 
different from TBE on 2 tests 

TBE/ESL no different from 
subme ra ion 

TBE better than submersion 



Math 



TBE no different from 
subme rsloa 

Submersion better than 
TBE in 1 of 3 grades; 
TBE no different from 
submersion In 2 grades 



Submersion better than 
TBE 



TBE no different 
from submersion 

Submersion better 
than TBE In 1 
of 2 graded; 
TBE no different 
from submersion 
In 2 grades 

TBE no different 
from submersion 



Olesini** 1971 



14 



Matching 



60 Spanish English 



(Continued) 



TBE better than submersion 
in 1 of 3 components 
of a standardized 
test; TBE no different 
from submersion in one 
component 



TBE better than 
submersion on 1 
component of a 
standardized 
test; TBE « no 
different from 
submersion on 
i component 
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TABLE _U SUMMARY W APPLICABLE STUDIES (Continued) 
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Author 



Pena- 
Hughes , 

Olkl 

Soils 
Plante 



Date Grade 



1980 



1976 



Ramos 1967 
et ah 



° SEDL 

(Stcb- 
bins) 



1977 



Skoczylas 1972 
Stern 1975 
2irkel 1972 



IG 



1-6 



K-3 



4-6 



1-3 



Design 



Number 2 
of 

Students 



Languages 3 
U L2 

; 



Random 
assignment 



156 Spanish English 



1-2 longitudinal; 72 Spanish English 

experiment 



Longitudinal; *** Hiligaynon English 
matching Pilipino 



Longitudinal; 1,060 Spanish English 
analysis of 
covariance 
and other 
adjustments ; 
5 sites 



Analysis of 
covariance 

Analysis of 
covariance 

Ha telling; 
analysis of 
covariance 



47 Spanish English 
213 Spanish English 
278 Spanish English 

(Continued) 



Reported Results 



L2 



Immersion better than 
TBE 



TBE better than 
submersion in 1 
grade. TOE no 
different from 
submersion in 1 
grade and for both 
grades combined 

TBE no different 
from immersion*,**** 



TUE no different 
from submersion* 



TBE no different 
from submersion 

Submersion better than 
THE* 

TBE better than submer- 
sion on 1 test; THE 
no different from sub- 
mersion on 4 tests 



Math 



TBE no different 
from immer- 
sion^, **** 

TBE no different 
from submersion 



Submersion better 
than THE 

Submersion better 
than TBE* 



* Treatment - 73, control not given. 

** Rejected for use of grade-equlvalenta only. 

*** Unable to obtain Information at preaent; however the s mp le size waa large. 

"l^^^S." «- *■ *** cannot be determined, but our beat 

2 l° t r * tudiea not »slng v .ndo« assignment, we note the method uaed to adjust for possible oreexistin, 

3 U is the language-minority child's home language; L2 la the child's aecond language. 

4 re,!o 8 r C ? 8Ult rePrCaent8 ^ C °" clusio " ^ "» •"»■"•• very complex analyaia; aee chapter 2 of the full 
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££2 S other -subject areas, which are often more depend- 

illJ^l t ?^ ^ " eh iS * " *' ( 1967) r *P°"ed the least 

filn ~? C ? f " ^J! 100 iQ ch « Uteracure. They found chat immer- 

2?!h^?? f 1 7" M e£fecCive ^ter 5 or 6 years as a T3E program in 
which all instruction was in U for grades 2 through 4, and in L2 thereafter. 

The data on ESL instruction are not very informative. As lust noted 

n?7 a T^% f ? Und K 3CrUCCUred inOerSi0n 3u ^« c ° ESL. Ames ana 3iSs 
\ ll ^Balasubramonian et al. ( 1973) found chat TBE programTwhich in- 

StJ !? ST£ We " n ° ^ than ESI, aloS Lum (W71) 

eijSe finding both that TBE programs which included an ESL 

component „ere no more effective than ESL alone and chat ESL alone was 

"Si, LegarT " a ( 1979) f0uad Ch " a TBE Prograf vitS Sa forked 
, batter than a TBE program without an ESL component. 

Mixed findings were found for several of the studies. As a result the 
reader will notice that there are nore findings than there are stu"es ' £j. d 
findings can b. attributed to different achievement results either from grace 

once^2%L r „?:r" n T?' *™ "udies ™7 be counted more San 

once as showing a positive, no different, or negative finding. 

With respect to TBE, positive outcomes pertaining to language perfora- 

197? ' J ^ 1 Plan " (1976) - Legarreta (197?), AIR 1975b), Cohen 

• U J* ZirkCl (1?7Z) - H0WeVer ' che «• f « the effec- 
±n ^nl f 1 ^i" 1 lnt0 I™"*™ scudies chat found Bo difference 
It J ST 84 P«f otmance between treatment and comparison groups (Ramos 
at al., 1967; Ames and Bicks, 1978; Plante, 1976; Kaufman, 1963; Huzar 1973- 

S^S Wt * , ,. l U 9s A * C ° hen ' 1975: SEDL ' 1977 = and Curtis Tsto- 

Tsil llltrJl ?Sr r S* , /? 72 io?f Sp * dd,B ' 1979 ' 19805 ^ub^nL etal., 
1973, Cottrell, 1971; Olesini, 1971; AIR, 1975b; Zirkel, 1972; Lum, 1971). 

or^Taum^^l;" TBE C ° ^ 1833 ef "" iVe *« eicherimmersion 

nL*M J ^ 1971;.Pena-Hughes and Soils, 1980) and some found TBE co have 

s2« ifSi m" b? co °P arison submersion (Danoff ec al., 1977, 1978? 

Scern, 1975; Moors and Parr, 1978; A. Cohen, 1975; McSpadden,. 1980)! 

TBE toiSS (197 J } ; i- C °5 en (1975) ' and A ° eS and Bicks < 1978 > found chat 
2L£*r^ a f?^ Ci ° n ° f Mth 3kms ' However, no effecc was found by 
J « L ^=^ l978) ' CaMrud and Curcis < 1980 >. and Parr (1978), 

(S'tI J Z 9 ' 19 *i°ol^' S 11 " (l975) ' C0Ve7 (1973) ' Olesini ( 1971 ) , SEDL 

97 ) <' S ^ la3 (1972) » «^padden (1980), and Scern 

U»/3; reporced a negative effecc. 

, am „^' U l i0n nust b « exercised in generalizing from cable 1 because some 
issues of methodological adequacy remain. For example, Covey (1973) and 
McCounell (1980a, 1980b) reporc success for programs' inclu^TBE nowever 
che Programs also included very low staf f-studenc racios~l co 8 in che pro-' 

co che^mall SEE?. ™ " ^ U " ° f ^truction or 



« i * a .^*°~ eKR5 < i in«l our findings to deteraine which studies would have beer. 

SesJnf- l7 1°°°^ ^ Cri " ria and aC « PCed ^de-equivalents. £ly ' 
Olesini would then be included in our results. His results were gene-ally 
favorable to TBE and have been included in table 1 and cable 2 gene ' ally 

studies' ^TSffi 1 " *< l0 ° k f " ? " Cerns in ch « fiad ±nss of all these 
!iSrS.r« S 2 *uaa*ri*M our findings with respect to comparing altema- 

^I'r 10 ^ approaches. We have grouped the 28 stuoies accoVding^o 
the comparisons they examine. Then, we have aggregated their findings ac- 
cording to whether che study had positive, no difffrence, or negitiS re- 

sol in "a^lfoL" *! ° C - e / a PP roach «* F « the first compari- 

f aC Ch8 of TBE versus submersion. For 

second-language acquisition, 10 .'innings favored TBE, 15 findings found no 

for TbT" 3 ° ™ 3uteersion ' and 5 flings were actLny^egative 

the rljJlf'^ Cab i 8 u 2 ""^ be I"* 1 " 1 *' *ather than simply counting 
SSbTST^ J " * eh Vari ° US ouCcoaes ' w * «" go beyond these tabula- 
tions and give more or le 3 s weight to different findings. For fflaapi! the 

B1 f ^ (WhiCh f0Und ChaC 188 £oduc£ belSlatn 
results than submersion did) took place in only one school -district li. 

£ SS-toViS^? 7 (Mhicfa fouad chac «^ 

J»3 designed to be nationally representative. Therefore, Danoff's findings 
S our ffSL 00031 ^^^ 7 r* WeiShC - . « clear undersong 

gat TLttfr SJ ^? °S 7 ^ ° b u Cained by l00kin « aC Che " udies th. aggre- 
m™. °° Wng at Che SCudi4ls ia i^^tion. Our policy implica- 

tions are presented below. «"p*ica 

Implications 

, J e believe the literature makes a compelling case that special programs 

s tt ^lT ^ Pr ° Ve Ch \ achieveo *« «" language-minority cSSren S« 

or preSrr!d b4 STT!' ^ 3 Pr0gran 3h ° uld be eiCher ^islated 

tLJ Z£ * 7 w 5 8deral GoverM »n«- Indeed, mora research and demonstra- 
tion projects with sound evaluation models are needed to determine whicTSr- 

SsToftMs " ° f ChUd " n in locations. Se ? 

rest or this summary will present our findings. 

Special Programs Can Improve Achievement in Language- 
Minority Students 

to ^r™ 6 ^"" WS J^ eW ? d lndicaces ch at special programs designed 
to overcome language difficulties in school can improve the achievement of 



notice J a i read7noCed ' f ne scudies "ad mixed results, the reader will 

1 V" are "re findings than there are studies. However, if 
a study administered five tests of which three had positive results 
and two -gative ones, we would record only one positive and one negative 
result i a our comparison tables. negaci/e 
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TABLE 2 



SUMMARY OF FINDINGS FROM APPLICABLE STUDIES* 
Transitional Bilingual Education Versus Submersion 



TBE: . Second 

Language Math 

Positive 

No Difference . l ° I 

Negative X \ 9 

Transitional Bilingual Education Versus En gi i3 h as a Second Ufag uage 

j3£ . Second 

* Language Math 

Positive 

No Difference .\\\\\\\\\ J J 

Negative . j ^ 

* 1 NA. 

Transitional Bilingual Education Versus Immersion* 

, Second 
' Language Math 

Positive...* 

No Difference, [[ ...... \\ ? ° 

Negative . 1 1 

1 0 

Immersion Versus English as a Second Language * 

* 

IMMERSION : Second 
" Language Math 

Positive 

i MA 



Math scores found in immersion projects in Canada are difficult to 
compare «ith scores in regular English curriculums. What can e conceded 
hoover, is chat students can achieve equally well (or better) in 
math classes taught in L2 as in math classes taught in LI. 
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language-^nority children. The studies ,by Pena-Hughes and Soils (1980, 

Q97n ™ C L (1976) ' J" 2 *" U973) ' Cov ^<^>> Kaufman (1968), and Lum 

, T «r ria * aCs ' attd «" *ho««d »P*cial programs to have pos- 
Mc^°Ji n nI 8G S f SS?' 1 The ingeaiOUS ^nexperiaencal design used by 
rf VSSrJif ' 1980b i^ so «•«"» CO have firmly established the presence 
iLlSS ^ r 0gr !f • £f ! Ce - Posiciv « «ff«ce« also were reported in the 
nonocp.ria.ntal studies of Zirkel (1972), Ames and Bicks (1968) AX* (1975M 

^ ii**!' N °"' Ch0Ugh « ChaC ^i 18 SP*^ 1 Programs have been 



The Federal Gover-na^r s hould Mot Place Exclusive Reliance on 
Transitional Bilingual Education 

<n.rJ° r <" 0r ?<f! Ma a deCadC ' Cfae Federal has worked toward 

institutionalizing transitional bilingual education as virtually the only 

2£°2h222 f Jff^SJ 0 ? ^^-minority children. l£ has ' 
n!S l P M Md ,. in T ^ Cl€ 711 fuadia * decisions. TBE has been implemented 
*l!n . * , b7 io h fl %° ff u iCe f ° r CiVil interpretation of ch. uTSS- 

tin^ V 1 ? ' ? e D *P» rtMac of Education .proposed, with feT^cceo- 

tions, the legal mandate of transitional bilingual education through Fed"- 

Pr0P ° 8al Chat b "" b ? che corrent Admin- 

M , *'? B * reviewed the literature on the effectiveness of transitional 
bilingual education we did not find justification for such heavy reliance 
on this method or instruction. In order for the Federal Government to 
rely exclusively on one instructional method for meeting the needs of 
language-minority children, the following two conditions must hold: 

1. There must be a strong case that the instructional 
method is '^iiformly effective. 

2. Effective instructional alternatives should not exist. If che 
desired outcomes can be reached through more than one aporoach, 
schools' GovernmenC shoula not constrain the options of local 

Only 28 studies that passed our methodological test addressed the ef- 
fectiveness of TBE, and only 11 of the 25 studies looking at TBE reported 
a positive effect. Further, additional methodological problems in Sese 
studies impose strong limits on generalizing their results. Three studies 



15 



i 



/ 

I 



suggMC chat the reported positive outcome could well have been due co 
other aspects of the program racher than co TEE icself (Covey, 1973- 
McCoun.ll, 1980a, 1980b; Plante, 1976). In addition, a^mber "studies 
chat used multiple-outcome measures found mixed results. Several other 
/.studies round a negative effect for USE when compared with submersion, 
°^ iaa *« ion (Dwoff et al., 1977; Moore and Parr, 1978; McSpadden 

" i!?L?f ,CB £» l972? C ° hen « 1975; Lua ' 1971 » S *™> 975; Pena-nughes"' 
and Soils, ,1980). Although we reviewed a limited number of immersion 
studies, each analysis of structured emersion generally found positive 
ma?S?InS?J apP " ach ' A «*i* v *~nt in "both language skill and subject 
!! TO S ^ g *7c S , b "" r Chr0U8h 3Cru ««red immersion than through ESL 
19/2; Pena-Hughes and Solis, 1980). 

These- findings do not add up to a very impressive case for the effec- 
tiveness of transitional bilingual education. We conclude chat TBE fails 
both tests for justifying reliance on it as the exclusive method for 

7 Ul * a *f***™* i *7 children. There is no firm empirical evidence 
that TBE is uniquely efxectiv. in raising language-minority students' per- 
rormance in tnglish or in nonlanguage subject areas. 

Since several States have followed the Federal lead in developir* pro- 
grams for language-minority children-in some cases, even legislating TBE — 
r analysis has implications beyond the Federal level. 

Federal Policy Should Be Flexible 

i ^ f" 00 " chan a decade, Federal policy (as expressed through Title VII 
legislation, Title VTI fun-ding decisions, OCR implementation of the "Lau 

SS B ?«.^fJ" A T St , 5 N0Ci " ° f ?ropoSed ^°aki"S) has emphasised 
transitional bilingual education to the virtual exclusion of alternative 
methods of instruction. We found through our analysis that this policy is 
SH£? ? * on the basis of educational effectiveness. While transitional 
ilingual education has been found to work in some settings, it has also 
been found ineffective and even harmful in other places. Furthermore, both 

fcJd W £^r nUCiV " !° ^-"^"ured immersion and ESL-have been 
rcund to work in some settings. 

The commonsense observation that children should be taught in a lan- 
V ^J^ md t ntiad d0eS noC necessarily i ea d to the conclusion they 
should be taught in their home language. They can be successfully taueht 
in a second language if it is done right. The key to successful teaching 
in the second language seems to be to insure that the second language and 
subject matter are taught simultaneously so that subject content never 
gets ahead of language. Given the American setting, where the language- 
minority child must ultimately function in an English-speaking society 
carerully conducted second-language instruction in all subjects may well 
be preferable to bilingual methods. 
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Vm conclude chac is is very hard co say what kind of program will suc- 

redtral policy is to allow schools to develop instructional programs chat 
suit the unique needs and circumstances 0 f their students. 

M «n2*r 11 n ° f*** 00 .' 0 aMUB » * priori chac che s *« approach chac is 
applied co a rural Souchwesc Texas discricc wich a large proporcion of 
second-gen.racion Hispanic children should also be applied Ma discricc 

h W as h b:.n tt haiJ r ° UP "J* r8fUgMS in * NorSh * ra Sc ledera! po Icy 

St I f r, SU f " ° v « c >« y»«. Our review indicated 

.hac a fundamencal change in Federal policy is needed. 

ZAuJti™*"? C ? iS ^f*" Wil1 rwluire "cognicion by che Department of 
tf££S 5 * P^fS 0 * 1 " 1 « choda Unguage-minority children can 

efrective and can meet civil right* criteria. Federal funding practices 
must ^cya t ,ch of che special programs designed Co SeclhJ 22 of 

ij^* gB ^ n0i: , it7 Cbildr * a " ChaC * aore rMlisc * c ^«ce among "So^s 
Frogram typ M is achieved. ^ »«.ous 

* Wi ? S ?f Md ACructured immersion demons tration program is especially 
wea^.™^? 0 "' Cht ^f 1011 °* chod h « b««n rejected on the basis of 
tit 5 •° r8Cical L a rguments.ll Immersion may not cransfer successfully ■ 

I 60 Che toiC,d SCaC "' buc chis **■-» empirical ques^^cnac 

imaediacely fund an excensive evaluacion of che McAllen, Texas, program, 
^,^V rU ' ^riaencal design for comparing che ef fecciveness of 
ec^oSc e ,c.^: rSi0a " d TBE f ° r M,XiCan - ABtriCan < SCuden " of lo » 

r^.! iV8 ^ Z u* f? aplexlt y of che Problem, ic also seems chac che Federal 
tlZTf S ? 0Ul Pr ° V . ide the " ,0,t CurrenC inioaaeion on pedagogical 
ISSIL J ^^S^^rity children so chac school discriccs can make 
informed choices, adapcing me c hods Co cheir local needs. 



Improved Bilingual Research and Program Evaluacions Arp m.^.^ 

More and beccer research and improved program evaluacions in bilingual 
educacion are necessary if che' needs of language-minority children are fo 
SceracSIl 7 «*" 1<W V** ° f Che ^^odology round Cnrougho" che 

S f T?^% P roblea ' tte »*Jor mechodological problems wich 

the literature include che following: 

o The absence of random assignment between creacaenc and control 
groups, 

<5 The use of scudy designs chac cannoc show a creacaenc effect" in che 
absence of random assignment, such as che norm-referenced model or 
failure co use analysis of covariance, and 

The failure co apply appropriaca scaciscicai cescs co taaonstrace 
program effeccs. 
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These problems have particularly characterized Tide VII evaluations. The 
Title VII bilingual program haa begun to take' steps to improve the quality 
of local results. However, our review has indicated that program evalu- 
ations are still of very poor quality; much improvement is still needed 
in this area. 

Bilingual education involves many "complex, difficult issues that have 
been little (or insufficiently) studied. Federal funding for research in 
the area of bilingual education was allotted for the Slat time under Part 
C of Title VII in 1978, with the Elementary and Secondary Education Act 
amendments (ESEA) • The need for additional research is gi s %at. 

Unfortunately, however, when Congress established the legislation in 
1978, it limited research to examining transitional bilingual education 
specifically, rather than all pedagogical methods for students with limited 
English "proficiency. A* a result, Federal research has been skewed to 
focus on one method. Ultimately, the development of effective instructional 
programs for language-minority children will come about only through a more 
broadly based research agenda. < 

Areas for redirected research should include the following: 

o A study of the divergent educational needs of language-minority 
children in the United States to include the examination of how 
these children's language deficiencies differ in their home lan- 
guage and English, 

o Examination of the effectiveness of alternative instructional ap- 
proaches and how these approaches meet the needs of different 
types of language-minority 'children, 

o A reexamination of the theory of TBE (designed for monolingual LI 
speakers), which may not be relevant to cany of the language- 
minority students in the United States, 

o Formulation of appropriate structured immersion curriculums, 

o Examination of the methods of English as a second language 
(vocabulary drills versus meaningful English communication), 
and 

o Examination of bilingual education teacher qualifications and the 
degree of fluency such teachers have in both languages. 
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NOTES 



I. The full report is found in Baker and de Kanter (1981). 

a f JL\i A i 1 ""^ 6 r8view 13 » secondary analysis limited by the level 
of detail the authors provide. This limitation was taken into account when < 
t drawing conclusions from the literature. 

rh. . 3 * J* 1 /^ 6 " C ° Che 01111(1,3 first ' or * om *> language. L2 refers to 
the second language, the language used by society and in the schools (in che 

SZiAl Y**^*?** 0 *** students). In the United States, L2 refers to 
S V°L he ^nguage-minoricy child; LI is chac child's home language 
(Spanish, Chinese, etc.). j.«tH»u* 5 e 

« 

4. Examples of studies rejected because chey used nonrandom assism- 

^rswK ^is*?* 8 souch san Francisc ° u979) « ^ 

phll . 5 ' f 0 ^ c*» Actors affecting che performance of language-ainoricy 
children in school, especially in learning English, are mecia^en 1979- 
Asher and Garcia, 1969; Giles, ,1971), differences in Haf^nglecween 
and written language skills (Cummins, 1978, 1980; Fishman, 1965), socio" 
lllll i <S5\"" ( ?°° r8 ' 19785 VelCaan ' 19805 ^"thal ec al., 197^^ 
TuM°^ C T iCiVC * bilicle< ' < Da "?. 1953; Peal and Lambert, 
MaSL^ 9 « ; „ Segal ?^ C2 ' 19755 H ««P te ^. 1977; Coronado, 1979; 

I 15465 Fi3ha *°. 1965; Jensen, 1963a, 1963b; Johnson, 1953, cited 

iSS r U 198 Vl PlaCC ° f bi » h ~^— or native-born 

cSSS t VI • }!# Kiaba11 ' 1968; Aaderson and john3 °°. i97i; 

SS i L Cardenas » 1972; Baral, 1979; Ferris, 1979), the de gree of 
tffl.^Hy ^T^" BaCtel « ^ 197 5>. ethnicity UosenLl ec al., 
de LSr "Soi) 1 he US™'. 198 °; Bala3ubrano nian •« I- 1973; Saker and 
Modi«£ 1973 ' ! denC S aocivaCion and 3elf-conceoc (Christian, 1976; 

1972^nJ JoJK ' i97 1 Z ' vonMa l ci,a . 1975; Del Buono, 1971; Skoc^las, 
»fj t ^ ' X ! ), .. parenCal SUPPOrt for che National program (Lambert. 

*, cL 2j ^ a "° n0> 19?1) ' and Ceteris tTcTtf-^com^n!" 
Ret? ^8^ 5 UBbfttt T uckar > 1^2; Skoczylas, 1972; 

In addition co chese background characceriscics of che child, numerous 

X I Sc^'Kf^VJ* 3Ch ° 01 ^ Che edu "^- a l Progr*. can arfect 
1«L r v^ 11 ?!^ 1 in3CniCCi °n (McDonald and Elias, 1976; Dulay and 

1979). 5 aCni2 " 19765 de KanCer ' 1979; Kramer, 

6. The following studias are examples of uses of a norm-referenced 
Sao?? 5 £r,{S5 ' I 0UQg U980) ' S " rn (1975 >' Co ^ 3 Christi" 1980a 
U3/8;, Arce (1979), Fairfax County (1980). 
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/ 7. The, following are examples of studies chat examined rains without 
any control groups: Guerrero (1980), Liberty Union (1980), Birm^hlm" 
Y^ 80 * 1 * (1971 >. Snith (1978), Cahill and Foley (1973), Ghini 

S?S ' t C ? h8n U971) ' SC " n (1975) « Pric8 < 1978 >> South San Francisco 
(1979),. Charlotte-Mecklenberg (1980), Arce and Sosa (1975). trancisco 

folloi'n^^L 1 " ° f .! he US * ° f Srade-equivaient scores can be found in the 
"llowing studies: Educational Management Services (1976), Young (1980) 

^Twfa^" 0 " (1979) ' GUes (1971) ' Trevino (1970), LufZl 1968) 
tillTJSnlk^ ? 978) ' 33113011 (1980) ' EUi * 8 " (1980t, Suth San ' 
Vairy1l9^: 9 ^97?5! e ^? e 9) g ^ ~ ^ ' 

slon £ 

of tL 0 *!^ 0 ^"/ ? h f * rai3ad ^" i00 « ^out the generalizability 
^ B " r !i n no£ 8S b * Sed ° n ^^^l^s Canadian children (see Tucker, 

tlllLjfrl ! r/ 19 5 ? " VieWed Che SCatUS of che Canadian literature and 

a 1.? immersion was applicable to children of lower socioeconomic 
status and to minorities. Pena-Hughes and Solis (1980) certainly indicates 

to JT ^u^ 18 in ChB Unl " d Scates ' but "««oh needs 

to be done because this question ultimately is an empirical one. 

• 11. See Tucker (1980) for a complete discussion of the inapplicability 
of immersion to the situation in the United States and Genesee (1976) for 
r<t 301 l 3ion u of u Che *™*UMlLty of immersion to the Canadian situa- 
tion. Pena-Hughes and Solis (1980) discuss a successful str^cturef Jm^er- 

socLe!^ Uni " d SCaC " * eh ^S^S^nority children of low 

socioeconomic status* 
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CHAPTER 1 

CONSIDERATIONS IN REVIEWING THE LITERATURE 



This investigation was begun at the request of .the White House Regulatory 
Analysis and Review Group for an assessment of the effectiveness of 
transitional bilingual education (TBE) . The request came during that group's 
review of the Department of Education's proposed bilingual regulations 
issued in August 1980. Although the new Administration withdrew the proposed 
rules, the question of the effectiveness of transitional bilingual education 
for language-minority students is still relevant for the development of 
Department policy. It is also a major issue for several States whose bilingual 
programs have followed Federal precedent. 

Since a comprehensive review of the literature on the effectiveness of * 
transitional bilingual education hai not been done for 5 to 6 years (Zappert 
and Cruz, 1977), a current review was necessary to meet the policy needs of 
the Federal Government. Such a review is a complex task, requiring not only 
the identification of studies bearing on our questions but also the application 
of standards of scientific research. These -standards provide a measure of 
the methodological adequacy of each study's approach and the extent to which 
the study results can be generalized into Federal policy recommendations. ' 
To enable the reader to follow how we carried out this task and to judge the 
validity of our conclusions, we have carefully defined our methodological 
criteria and illustrated how we applied these criteria in the review procedures. 

In the remainder of this chapter, we will summarize the history of the 
Federal Government's involvement in the issue of the civil rights of 
language-minority children, describe alternative methods of instruction, 
explain how the review was done, and discuss the major methodological problems 
found in the studies. 

History of Federal Involvement in Civil Rights Issues for Language-Minority 
Students 

Data collected by tht Federal Government and private civil rights and 
educational organizations in the late 1960 's revealed substantial evidence of 
discrimination against language-minority students, especially Hispanics, in 
the Nation's public elementary and secondary schools. Statistics on 
academic achievement and school retention documented the effects of this 
discrimination, It was clear that hundreds of thousands of language-minority 
students suffered severe academic retardation and exceptionally high dropout 
rat es. 

While conducting compliance reviews, the Office for Civil Rights of the 
Department of Health, Education, and Welfare (DHEW) discovered a number of 
common practices that had the effect of denying equal educational opportuni- 
ties to language-minority students. These practices related to the way in 
which schools responded to the English language skill deficiencies character- 
istic of many language-minority students. 
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On the basis of this evidence, and using its authority under Title VI of 
the Civil Rights Act, the Office for Civil Rights sent a memorandum to school 
superintendents on May 25, 1970, "to clarify DHEW policy on issues concerning 
the responsibility of school districts to provide equal educational opportun- 
ity to national origin/minority group children deficient in English language 
skills." The memorandum stated in part: 

Where inability to speak and understand the English language excludes 
national origin/minority group children from effective participation 
in the educational program offered by a_ school district, the district 
must take affirmative steps to rectify the language deficiency in 
order to open its instructional program to these students. 

The memorandum required that— -< 

o School districts not use English language ability as a basis 

for assigning national origin/minority group students to classes for 
the mentally retarded or to deny these students access to college 
preparatory courses "on a basis directly related to the 
failure of the school system to inculcate English language skills." 

o "Any ability grouping or tracking system employed by the 
school system to deal with the special language skIII needs 
of national origin/minority group children must be designed to 
meet such language skill needs as soon as possible and must 
not operate as an educational dead-end or permanent track." 

o School districts must notify the parents of national origin/ 
minority group students of the school activities that are 
called to the attention of other parents. The notice, to be 
adequate, must be in a language they can understand. 



Lau v. Nichols Case 



The Office for Civil Rights memorandum was affirmed by the Supreme Court 
in its 1974 decision in the case of Lau v. Nichols. Lau was a plass-action 
suit against the San Francisco Public School District which alleged that 
the district's failure to provide special educational services to non-English- 
speaking Chinese students violated both the equal protection clause of the 
14th Amendment and Title VI of the Civil Rights Act of 1964. 

In Lau, the Court reviewed the California Education Code and concluded 
that— 

Under these State-imposed standards there is no equality of 
treatment merely by providing students with the same 
facilities, textbooks, teachers, and curriculum; for students 
who do not understand English are effectively foreclosed from 
any meaningful education. 
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Basic English skills are ac the very core of what these public 
schools teach. Imposition of a requirement that before a child 
can effectively participate in the educational program, he 
oust already have acquired those basic skills is to make a mockery 
•of public education. We know that those who do not understand 
English are certain to find classroom experiences wholly 
incomprehensible and in no way meaningful. 

It seems obvious that the Chinese-speaking minority receives 
less benefits than the English-speaking majority from respondents 1 
school system which denies them a meaningful opportunity to 
participate in the educational program — all -earmarks of the 
discrimination banned by the Regulations, * 

The Court declined to rule on the constitutionality of the school dis- 
trict's program, focusing instead on the statutory prohibition against natio- 
nal origin discrimination set out in Title VI of the Civil Rights Act of 
1964, Accordingly, the Supreme Court found the San Francisco Public School 
District to be in violation of Title VI. However, the Court declined to 
prescribe a specific program that would provide equal educational benefits, 
stating: 

Teaching English to the students of Chinese ancestry is one choice ♦ 
Giving instruction to this group in Chinese is another. There may 
be others. 

The precedent of relying only on transitional bilingual education to 
meet the legal requirement was established by the Federal Government in 
implementing Title VI of the Civil Rights Act, 



Lau Remedies 



Following the Supreme Court's decision in Lau , the Department of Health, 
Education, and Welfare assembled a group of outside and departmental 
education experts to develop informal policy guidelines outlining the remedi- 
al responsibilities of school districts that failed to comply with Title VI 
and the principles enunciated in Lau . This group produced a document enti- 
tled "Task Force Findings Specifying Remedies Available for Eliminating Past 
Educational Practices Ruled Unlawful Under Lau v. Nichols "— better known as 
the Lau Remedies. 

The Lau Remedies outlined the major elements that should be included 
in school districts' plans to remedy Title VI Lau violations* According to 
the Lau Remedies t compliance plans should, among other things, provide for 
the following: 
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e_ Identification of students with a primary or home language other 
than English. . 



o Assessment of the relative proficiency of such students in English 
and their native language. 

o Instruction of elementary students through their strongest language 
until the students are able to participate effectively in a classroom 
where instruction is given exclusively through English. 

o Provision of special language instruction and compensator/ educa- 
tional services to secondary school, language-minority students who 
are underachieving academically. 

Because the Lau Remedies were never published as proposed regulations, 
their underlying assumption that TBE was the best, if not the only, way to 
satisfy the civil rights- requirements was never opened to public debate . 

A cover letter transmitting the Lau Remedies to school officials ex- 
plained the document's legal application. In part it stated: 

Voluntary compliance plans which set forth educational strategies 
consistent with the approaches outlined in this document and which 
contain the other elements specified therein, will be accepted by 
this office. School districts submitting voluntary compliance plans 
to this office which are not consistent with the outlined approaches 
or with other required plan elements must demonstrate affirmatively, 
at time of submission, that such plans, at a minimum, will be equally 
effective in ensuring equal educational opportunity. 

Thus, the Federal Government placed the burden of proof on the schools 
to demonstrate that an alternative to TBE was effective, even though the 
Government had never shown TBE to be effective . 

Although DHEW used the Lau Remedies to negotiate numerous voluntary com- 
pliance plans, the document's legal authority was challenged in a 1978 Feder- 
al court suit, Northwest Artie v. Calif ano . As a result of the suit, DHEW 
agreed to publish its Title VI Lau compliance standards in the Federal Regis ter 
for public comment. In keeping with the court-approved agreement, the Department 
of Education published a Notice of Proposed Rulemaking August 5, 1980. 

The standard proposed in the notice required that transitional bilingual 
education be used to meet the needs of all language-minority students identi- 
fied as eligible for services. It called for special instruction to encour- 
age fluency in English while other content areas of the curriculum would be 
taught in the child's home language until that child mastered English well 
enough to succeed with all instruction in English. 

Transitional bilingual education is only one of several instructional 
methods under the generic rubric of bilingual education that attempt to meet 
the needs of language-minority students. Because alternative instructional 
approaches are available for meeting the educational and civil rights needs 
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of these children, transitional bilingual education should clearly have been 
Icaowu co be superior co che alternatives when the Federal Government , 
attempted to mandate use or this method in the schools. 

Following an extended period of public comment oti the proposed regula- 
tions, the notice was formally withdrawn February 2, 1981. Although the 
proposed rules were withdrawn, the question of the effectiveness of tran- 
sitional bilingual education is still important for the following reasons: 

o The Department may still be under a consent decree requiring that the 
Lau Remedies be replaced with formal regulations, so new regulations 
may have to be drafted. If a particular instructional approach can be 
justified, mandating it vould be one option considered in the develop- 
ment of new regulations. 

o Withdrawal of the Title VI Language-Minority Notice of Proposed Rule- 
making and the absence of an alternative leaves the Lau Remedies 
in force. The past practice of the DHEW Office for Civil Rights (sow 
in the Department of Education) in developing compliance agreements 
has stressed transitional bilingual education through the Lau 
Remedies. Therefore the issue of the effectiveness of transitional 
bilingual education is as appropriate in assessing the Uu Remedies 
as in assessing the Notice of Proposed Rulemaking. 

o Other Departmental policies and programs—especially Tide VII funding 
practices— have strongly emphasized transitional bilingual education 
to the exclusion of alternative methods of instruction. In fact, the 
Department's whole approach to the problem of rhe language-minority 
child over the past decade can fairly be characterized as being based 
on the assumption that TBE is, with rare exceptions, the only accept- 
able approach. It is worthwhile to reassess this assumption- by re- 
viewing the evidence on TBE effectiveness. 

o Federal policy has formed the basis for bilingual programs and legis- 
lation in several States. When States follow the Federal lead, both 
the States -*nd the Federal Government need to be sure that the path 
taken by Federal policy is justified. 

Types of Programs foe Language-Minority Students 

A number of schemes "for-classifying types of bilingual instruction exist; 
Valencia (1969), for example, identified 19 different models of bilingual ed- 
ucation (also see Pauls ton, 1975). For our purposes three models identified 
in the literature are sufficient to compare with the "submersion" method— 
that is, doing nothing for the language-minority child. 

In a submersion progra^* language-minority children are placed in an 
ordinary, English-speaking classroom with no special program to help them 
overcome the language problem. Submersion is aptly described as "sink or 
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swim." The child J Trhome language (U) is not used in submersion.* The 

Supreme Court's Lau decision in effect outlawed submersion programs in the 
United States. 

The three alternative methods chat are not ruled out by the Lau decision 

are the following; 

1 « EngUsh as a Second Language (F,SL) . In an English as a second lan- 
guage program, language-minority students are placed in regular instruction 
.ormost ot the day. During some part of the day, however, their curriculum 
differs from that of the regular classroom in giving extra instruction in 
mastering English. Generally, this extra help is based on a special curric- 
ulum designed to teach English as a second rather than as a first language. 
LI may or may not be used in ESL instruction. 

2 - Transitiona l Bilingual Education (TBE ). Subject matter in a trans- 
itional bilingual education program is at least partially caught in the home 
language (LI) of language-minority children until their second language (L2) 
^English) is good enough for them to participate successfully in a regular 
classroom. English as a second language is often used in conjunction with a 
transitional bilingual education program to help minimize the time the children 
spend in mastering English. It is also generally held that learning to ' 

read in LI facilitates learning to read in L2. Jome times LI instruction 

is gradually phased out and regular English instruction is gradually "phased • 

in; in other cases, the change is more abrupt, with students being mainstreamed 

out of the LI program. The ultimate goal of transitional bilingual education 

is to move students into the all-L2 program. This method is differentiated 

from the submersion and English as a second lar^uage methods by the use 

of LI for instruction. 

3 - Structure d Immersion . In a structured immersion program, almost all 
instruction is given in L2. There are, however, fundamental differences 
between structured immersion and submersion. First, immersion teachers are 
bilingual. Second, students can ask questions of the teacher in LI, although 
the teacher generally replies only in L2. Most important, the curriculum 

is structured so that it does not assume prior knowledge of the second language 
when subject areas are taught. All content is introduced in a way that can ' 
be understood by the students. The students, in effect, learn L2 and 
content simultaneously. Immersion differs from bilingual instruction in 
that it deemphasizes home language use by the teacher and it gives subject 
area instruction in L2 from the beginning. Structured immersion programs may 
include a period of LI language arts during the school day. 



* LI is^an abbreviation for the child's first learned language. L2 is the 
child s second language. In the case of language-minority groups in which 
some children could have learned both languages from birth, LI refers to 
the non-English (minority culture) language and L2 is the normal language 
of schooling and of the majority culture (i.e., English in the United 
States). 
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These chree program descriptions are, of course, ideal types. In prac- 
tice, ic is sometimes hard to classify a particular program as one or anoth- 
er. Furthermore, each of these general models can encompass a considerable 
SsSnctionj^ 71 ' 1 "* NeV * rthelMS ' che " «*P«« have real and s';ni£icaat 

!? rh u?f Che u besc Co "nceive of these models is as different educa- 
tional philosophies chat guide che development of specific programs. What 
is considered appropriate or inappropriate for a program will vary, depending 
on whicn one of chese philosophies is followed. The models correspond to 
real program distinctions recognized in che literature, since w derived 
chem largely from the way the literature conceptualizes prograranatic 
differences. ° 

Before proceeding, we should note that the three instructional models 
and submersion differ on both civil rights and educational dimensions. 
Submersion is che lack of a special program which the Supreme Court found to 
violate the civil rignts of language-minority children. The other three 
methods are alike in seeking co correct the civil rights problem described 
children!"" ~ deci3ion by P rovid i n S special help to language-minority 

m The differences among che chree mechods can be illuscrac'ed by an outline 
or -che. argumencs -advanced as co why each mechod should succeed in solving che 
problems of language-minoricy children: 

0 f°^ i3h /? a Second Language: Concencraced , supplemencary inscruccion 
in EngUsh Unguage skills will enable scudencs co learn English fasc 
enough co keep up wich EngUsh-s peaking ceachers in che various 
subjeccs. 

0 transitional Bilingual Educacion: While che children are learning 
English, chey should be caughc subjecc macerial in cheir home language 
so cheir academic progress will noc be recarded by cheir lack of 
EngUsh skills. Ic is easier for che language-minoricy child Co 
firsc learn co read in che home language racher chan in English. 
Furcher, firsc learning co read in che home language will facilicace 
learning co read in English. 

o Scruccured Immersion: The solucion co developing scudencs' EngUsh 
while chey progress in ocher subjeccs is co ceach all subjeccs in 
cnglish ac a level underscandable co che scudencs. Alchough che 
curriculum assumes no prior knowledge of EngUsh, language-minoricy 
scudencs in effecc learn EngUsh as chey learn mach and learn mach 
through EngUsh instruction that is understandable at their level 
of EngUsh proficiency. 



The Plan of This Review of the Literature* 



Although many of the studies we reviewed e xam ined several outcome mea- 
sures, we are limiting the discussion in this report to two questions: 
Does bilingual education lead to better performance in English? Does it 
lead to better performance in nonlanguage subject areas? Our decision 
to concentrate on English and subject matter acquisition stems from basic 
Federal policy that recognizes the need to prepare language-minority child- 
ren to function successfully in an English-speaking nation, and seeks Co 
provide equal educational opportunity to them. 

A program that produces mediocre English performance while maintaining 
the home language skills will be judged a worse program than one chat pro- 
duces better second language performance while ignoring home language skills* 
The juscificacion for chis viewpoinc is chat, in che Uniced Scaces, any 
successful educacion program muse prepare che scudencs Co participaca in an 
English-speaking society. Therefore, che overriding concern in evaluating 
instruction for bilingual scudencs is how well chey learn English, 

A number of ocher goals have been puc forward for bilingual educacion: 
less absenteeism, lower dropouc races, improved self-concepcs and accicudes 
coward school, and developmenc of bilingual adulcs. Any attempt to sys- 
cemacically address all chese goals is beyond che scope of chis report. We 
will, however, from time to time call the reader's attention to certain 
interesting findings regarding these other goals. 

This is the most comprehensive review of the literature on bilingual edu- 
cation to date. Studies were identified by ERIC search, by consultation 
with experts in the field, from prior reviews, and from lists of studies. 
In each study we reviewed, we looked at the bibliography in an effort to 
identify additional studies. 

We reviewed more than 300 documents (see attached bibliography). Of 
these, about 150 were program evaluations. In addition, Zappert and Cruz 
(1977) reviewed 175 studies, mostly Title VII evaluations which the Title VII 
program office can no longer produce. We did manage to locate most of che 
12 methodologically acceptable studies Zappert and Cruz cited, and we accept 
their judgment that the rest were unsound. 

Most, but not all, of the studies we reviewed are Title VII evaluations. 
They cover every region of the country, almost every State, rural and urban 
areas, migrant and nonmigrant students, and a variety of language groups. 
Some studies come from other countries. 
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Methods Used co Assess che Studies 



Once we deterained che focus of che review, we read some 300 documents 
looking for answers to our questions. A study may be of no use in answering 
these questions for either of wo reasons: First, it may be looking at different 
questions, and so it simply does not apply to ou concerns. A number of 
studies address specific local Issues that do no. generalize to our questions. 
Therefore, we were unable to sake any further use of these studies. Second, 
flaws in a study's methods may raise doubts as to whether the reported program 
outcome might have been the result of something other than the intended 
effect of the program. Thus, the initial review task was to decide which 
studies are applicable to the issue and which studies cannot be used because 
of methodological problems. 

The approach used in this review was based on the application of standard 
methodological criteria for the adequacy of research designs which are widely 
accepted in the education literature. These criteria were applied to the studies 
being examined to see if they were of acceptable quality. In this review, the 
criteria for methodological soundness were applied in a way that recognized 
that a design weakness in one area can be compensated for by strength in 
another area. Previous reviews have not recognized such complexities. For 
example, Zappert and Cruz (1977) rejected the study by Covey (1973) for 
failing to control for initial differences in language ability. However, 
because Covey f s study was a true experiment, no further control of language 
ability was necessary. 

The list of possible methodological pitfalls is long, and we did not 
require that a study pass every hurdle. Rather, we looked for fatal flaws 
in the study. A fatal flaw can be a single problem, such as using a post test-only 
design without random assignment. The flaw also may be the cumulative impact 
of a number of separate problems. 2 

The basic objective of scientific research is to rule out alternative 
explanations^ Scientific research is easier to describe in theory than it 
is to put into practice, however, and few, if any, studies succeed in completely 
overcoming all possible methodological problems and in eliminating all possible 
alternative explanations. The reviewer must exercise professional judg- 
ment as to whether there is a reasonable possibility that the authors of the 
study ruled out alternative explanations. * 

For this review, the methodological problem was to be sure that the observed 
results were the consequence of the program under stua/ and that alternative 
explanations for the results can be ruled out by virtue of the study design. 
The best method for achieving this goal is an experiment in which random 
assignment is used to select two groups of students, one of which receives 
the usual school program while the other receives the special treatment 
(instruction in English as a second language, immersion, or transitional 
bilingual education). Performance of the two groups is then compared by 
statistical methods which enable us to estimate the probability that the 
observed differences between the two groups were not due to chance. 




Although experiments are rare in educational research, ve did locate 
six true experiments ia our literature search. In the absence of a 
true experimental design, the problem of carrying out a study chat eliminates 
alternative explanations becomes more difficult "since there are a variety or 
£ESL°a f?% Siblt alC ? n4Cive «Plan.tions. The methodological gSssarJ in 
enSunterti^ "" 7 ° f *" P °" iblt ^^i^l problems thatmight* 

We found the following kinds of studies to be clearly acceptable: 

° ^^ ?eria " n x S (WiCh rand °° —l&mnz between treatment and 
control groups). 

o Studies using nonrandoa assignment in which researchers had aade some 

? ^ !;!!f e ?•? CS, €i ^ tr chrou S h etching or through statistical ad- 

l£2Ei2"!!£? 7 4Ml7SiS ° f covasi4ac *>' " control for possible 

5t?2V? ?? C ^o^ CWeen Cht gr0Ups ' (See Uob «" and Tucker 
(1972) and McConnell (1980a, 1980b) for examples of how good desire 
can overcome the problems of nonrandom selection.) 

mozll Ittot/* ^ Ced Ch *, C We u did n ° C ^^eically discount all nonexperi- 
^ental studies. We recognize that both matching and analysis of covariance 

S T r ^ 7 " iZU * d f ° r f * Uing " ov « c «" P"blems created by 
nonrandoa selection. While these criticisms are certainly valid "least 

{L T "!?V tae CrUt «P« rl « n « «• d«rly superior to the alttrnatJves 
m^imT^ "i"? St i d °° ° tC in Cht " 4l *• «*« of educaSonal Valuation 

SSn?£of thTtrob?^ J—* 1 "" S«n«ally accepted methods of cor- 
There??,-. J ! P ' ? f nonrando ° ••l.ccion, other things being equal. 

So°i r o e gic W all d ; s^und 1 ! 51 " 00 — °^ ~ « 

^n.I^fL'f" methodological assessment determined the limits of 

drawn « ^ 7 °S ? CUd7 ' S rMUlCS and che ^P^cations that could be 
drawn from them. Obviously, a study's results apply to the particular 
group of students studied, but this information alone is not very useful. 
Ultimately we want to know if cb e condu.iocs apply to all lanJSge^minority 
students or only to some particular segment. nguage-minority 

Results can acquire wide generalizability ia two ways. First, the stu- 
entire"^ ? J" 1 ^ * Wy M <° * "Presentat^e of he" 

SLSeT*?}?? 0 " 5* S u CUd * n " " *" this case, language- 

2^SfL21£S ^f^"* StaC8 V °V? 1 of che 3 °0 documents 

wt reviewtd falls into this category (Danoff cc al.. 1977 1978} <z**~„a 

consistent findings in many different setting, ca^ be the 'basis for genelal- 

iaiag. Thus, if every study cam. up with the same, result, no matter tow 

limited in gen.ralizabili ty each individual study was, the weight of The 

collective evidence would be compelling. 
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' Reasons for Rejection of Studies 

We considered any of the following characteristics sufficient cause to 
reject a study as unsuitable for our purposes: 

1. Failura to address the issues we are considering here, 

2. -Nonrandom assignment with no effort to control for possible initial 
differences between control and program groups, 

3. Norm-cefereuced design, 4 

4. Comparison of "posttest" scores only, with nonrandom assigtanent,5 

5. Reliance on school-year gains for the program group without 
a control group, or 6 

6. Reliance on grade-equivalent scores.? 

We have already discussad the first item. Discussion of the remaining items 
f o Hows • 

Nonrandom Assi gnment Without Control for Possible Initial Group Differ- 
ences. The basic problem in assessing a study in which random asaignmant 
was not used is to insure that the group exposed to the treatment does not 
differ from the control group on some other variable that affects learning. 
Selection bias is a possibla consequence of nonrandom assignmant of pupils 
to the control and program groups; the supposed observed outcome of the 
program may simply result from original differences batweeu the two groups 
on some characteristic related to achievement. 

Our first step, therefore, was to identify what other factors are known 
to affect the learning process of bilingual students. Among the factors 
affecting the performance of language-minority children in school, especially 
in learning English, are the following: 

o Age (Krashen, 1979; Asher and Garcia, 1969; Giles, 1971); 

o Differences in learning between oral and written language skills 
(Cummins, 1978, 1980; Fishman, 1965); 

0 Socioecono mic 3tatus (Moore, 1978; Veltman, 1980; Rosenthal et al*, 
1978; de Avila, 1981); 

o Ethnicity (Rosenthal et al., 1981; Matthews, 1970; Veltman, 1980; 
Balaeubramonian et al., 1973; Baker and de Kanter, 1981);. 

0 Student's motivation and self-concept (Christian, 1976; Modiano, 
1973; Zirkel; 1972; van Maltitz, 1975; Del Buono, 1971; Skoczylas, 
1972; Rand, 1980); 




0 Parental support for the educational program (Lambert and Tucker 
1972; Del Buono, 1971); 

0 Character istics of the community (Lambert and Sidoti, 1980; Lambert 
and Tucker, 1972; Skoczylas, 1972; Read, 1980) 8 ; 

0 Various cog nitive abilities (Darcy, 1953; Peal and Lambert, 1962; 
Landry, 1974; Segaiowitz, 1975j Humphrey, 1977; Coronado, 1979; 
Malherbe, 1946; Fishman, 1965; Jensen, 1962a, 1962b; Johnson, 1953, 
cited in Albert and Obler, 1980); 

0 Place of birth— immigrant or native-born (Carter, 1970; Troika, 1978; 
Kimball, 1968; Anderson and Johnson, 19-71; Cardenas and Cardenas, 
1972; Baral, 1979; Ferris, 1979); and * 

0 Dagrae of home language dominance (Battel et al., 1975). 

In addition to these background characteristics of the child, numerous factors 
associated with the teacher, school, and the education program can affect 
the outcome of bilingual instruction (McDonald and Elias, 1976; Dulay and 
Burt, 1979; Engle, 1975; Patniz et al., 1976; de Kanter, 1979; Kramer, 1979). 

The procedures used to assign students to bilingual programs can introduce 
bias according to student characteristics. For example, bias occurs when 
parents are permitted to volunteer their children (nonrandom assignment) for 
a special bilingual program. Parents wfeo volunteer children are usually 
more involved in their children's schooling than parents who do not volunteer 
their children. They may provide more help and encouragement < to the children 
in their school work than do .the parents who remain silent. Moreover, superior 
students are likely to come from a home environment in which the parents are 
actively involved with their children's schooling. We must suspect that 
volunteered students are likely to be better students than other children 
are. Thus, the prograa,may show "gains" due to the inclusion of better 
students even though the program is in reality no more effective than regular 
schooling (see Laumann, 1969). 

Another possible bias introduced with volunteered students is that 
parents of children with an unusual gift for languages may yant those children 
to benefit from a special language program. Again, students' progress may 
have little to do with the specific program— rather , gifted students would 
stand out in any* language program. 9 

Graduation from the program also may introduce a selection bias. If 
schools mainstream students as they reach some level of performance, students 
who perform relatively poorly will accumulate in the program for two reasons: 

o Poor performers will stay in the program longer. 

o The better performers who leave will, on the average, be replaced 
with students who are poorer performers than the graduates were. 
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Compariso n With Norms . When the effaces of a program are evaluated, the 
performance of students in the program must be compared with the performance 
of a similar group of children not in the program. If random assignment is 
not used, it becomes very difficult to insure that the comparison group is 
similar. One approach to the problem is the norm-referenced model, in which 
^ the performance of students in the^ptogram is compared' against national 
norms by measuring fall to spring gains in percentile scores. This is by 
far the most commonly used model in Title I evaluation (Goor et al., 1980). 
However, the applicability of the norm-referenced model in the evaluation 
of bilingual programs is questionable. 

When the norm-referenced model is used in an evaluation, it is assumed 
that the expected rate of improvement of students in the program would have 
been the same as that of the norming group in the absence of the special 
program. ?or several reaisons., this assumption is probably not valid for 
language-minority children. 

The nature of the learning curve for language-minority children is not 
known. It is often assumed (see Sgan and Goldsmith, 1981) that, in the 
absence of special help, these children will fall further behind the norms 
over time, since they cannot understand instruction as well as do the mono- 
lingual English-speaking children upon whom the norm is based. We found 
reason to question this assumption, however. Using a nationally representative * 
sample, we found that when standardized vertical-scale scores were examined 
over a 3-year time period the language-minority students began below the 
monolingual English-speaking group but did not fall further behind over 
the 3 years. Although more research is needed on this point, it calls 
into' question the assumption that the performance of language-minority 
children gets worse as they mature (compared with the performance of 
monolingual Esgiish-speeking students). 

A second problem of the norm-referenced model is that, as the monolingual 
non-English-speaking limit is approached, an achievement test becomes both a 
test of communication in English and an achievement test. If students know 
the answers to the .questions but cannot understand the test, their scores 
will be loy. If they then learn enough English to be able to understand the 
test, thnir .scores will rise because they now can communicate to the test 
what they know; Therefore, they will register large gains on tt\e test even 
though they have not increased their knowledge of what the test purports to 
* be measuring. It is possible, therefore, that small increases in English 
5 skills will translate into large gains on the test for the initially lowest 
scoring students (evidence of this phenomenon was found in Garcia, 1979; 
Young, 1980; Cohen, 1975; Stern, 1975). 

We believe this phenomenon accounts for the spectacular gains of percent- 
ile scores, especially in math, occasionally reported for bilingual students. 
It is not so much that they learned better when instructed in the home lang- 
v uage as it is that they learned enough English during th* school year to be 
able to communicate to the test what they already knew. If this analysis is 
correct, then any use of the norm-referenced model in evaluating bilingual 
programs is highly suspect. 
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Finally, in the Lau decision the Supreme Courc ruled chaC submersion is 
an unacceptable educational approach for language-minority children. There 
is no question that submersion students will eventually learn English. The 
issue is that it takes Coo long and that more effective methods of English 
instruction should be employed. Therefore, the appropriate standard of com- 
parison in bilingual program evaluations is the performance of submersion 
students, not the monolingual English-speaking norm. ?or these reasons, we 
decided norm-referenced designs were not acceptable in addressing the issues 
vich which we are concerned. . 

Comparison of Posttest Scores Only, With Nonrandom Assignment . Some 
studies comptre only posttest scores of students in the program and of a 
nonrandomly selected comparison group. This design is acceptable in a true 
experiment, since the random assignment of students insures the comparability 
of the experimental and control groups. However, if assignment was not 
random, this design does nothing to take into consideration preexisting 
differences that could lead to differential performance between the experimental 
and control groups. Therefore, this approach does not constitute an acceptable 
evaluation design. 

School-Year Gains Only, Without a Control Group . Several evaluations 
report only the differences between the program students' fall and spring 
scores, even testing the gain for significance. This procedure is unsound. 
Almost all students show some absolute gain over time, even if they are at 
the same time rapidly falling behind the norm. Consider the following hypo- 
thetical data: 

SCORE 
Fall Spring Gain 

Program 100 125 25 

Control 100 150 50 

According to evaluations that consider only program students,, the gains 
would indicate program success. It is clear, however, that since the con- 
trol group gained more than the program students, the program was far from 
effective. The* point is that a simple examination of gains over the school 
year for students in a special progranryields too little information to< 
permit determining whether the program worked. Researchers must also compare 
the progress of program students with the normal rate of progress made by 
students not in the special program. Hence, a study design that examines 
gains over the school year without a control group is unacceptable. t 
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Grade-Equivalent Scores . Studies chat are based on grade-equivalent 
scores pose serious problems. Grade-equivalents do noc correspond to the 
time pattern of learning and the methods used to produce them may not repre- 
sent equal learning. Evaluation experts have often criticized use of grade- 
equivalents. To quota from *A Prototype Guide to Measuring Achievement Level 
and Program Impact on Achievement in Bilingual Projects" (Horst et al.; 
1980): 

They are based on the mistaken belief that a gain in test scores 
of one or more months for each month of instruction represents] 
good progress. This is not true. Grade-equivalent scores provide 
an illusion of simplicity but, in fact, they are almost impossible 
to interpret, even for specialists in test construction. Grade- 
equivalent scores should never be used by anyone for any purpose 
whatsoever, (emphasis added) ' 

~* 

However, there is disagreement among testing experts whether grade-equiva- 
lents are totally unacceptable for measuring student achievement. One study 
(Olesini, 1971) which would have been accepted except for its use of grade- 
equivalents has been separately identified. 

Outline of This Report 

The remaining chapters of the report discuss the application of the 
accepted standards of scientific research (see Campbell and Stanley, 1963, 
for a partial list) to the literature on the effectiveness of bilingual 
education. Chapter^ discusses the studies we found to be applicable to 
our questions, focusing on what limits are imposed" in generalizing f roo ■ 
these studies. Chapter 3 discusses why we found a number of studies- 
including several studies widely cited by proponents of transitional bi- 
lingual education as evidence of the effectiveness of such education — not 
to be applicable to our concerns. Chapter 4 presents our conclusions. 
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NOTES 



1. Because a literature review is a secondary analysis, and the level of de- 
tail provided by the authors varies, we are limited as to what can be learned 
from the studies reviewed. These limitations were taken into account when we 
drew conclusions from the literature. 

2. The methodological criteria we consider are generally accepted 
principles of social science research, such as those discussed by Campbell 
and Stanley (1963). 

3. Any review is based on professional judgment, although this fact is 
often not stated. People who disagree with our findings may argue that we 
applied arbitrary criteria and personal judgment. Therefore we have care- 
fully explained our methodology so that readers can assess for themselves 
how we made our scientific evaluation. Our effort to ^explain this process 
makes the p^per long. 

4. Studies that, used a norm-referenced design include ' Rimm, 1980a; 
Young, 1980; Stern, 1975; Corpus Christi, 1980a, 1980b; JDRP, 1977b; St. 
John Valley, 1980; AIR, 1975e; Ames and Bicks, 1978; Arce, 1979; Fairfax - 
County, 1980. 

5. The following are examples of studies chat were given no further 
consideration because they used a posttest-only design: South San Francisco, 
1979; Elligett, 1980; AIR, 1975e, JDRP, 1977b; St. John Valley, 1980; Del 
Buono, 1971. . 

6. The following studies looked at "pre-post" gain with no control 
group: Guerrero, 1980; Liberty Union, 1980; Birmingham, n.d.; Valencia, 
1971,*>Smith, 1978; Cahill and Foley, 1973; Ghini, 1975; B. Cohen, 1971; Stem, 
1975; Price, 1978; South San Francisco, 1979; Charlotte^Mecklenberg, 1980; 
Arce and Sosa, 1975. 

7. Studies that used grade-equivalent (GE) scores include Educational 
Management Services, 1976; Young, 1^80; External Evaluators, 1979; Giles, 
X971; Trevino, 1970; Kaufman, 1968; Olesini, 1971;-Leyba, 1978; Hanson, 
1980; Elligett, 1980; South San Francisco, 1979; Charlotte Mecklenberg, 
1980; JDRP, 1977b; St. John Valley, 1980; AIR, 1975e, Arce, 1979, 

8. The glossary in appendix A gives more detail on how these variables 
affect learning in bilingual education. 

9. We should also note that equating the treatment and comparison groups 
for IQ or initial achievement probably does not account for this type of 
ability. We encountered no evidence in our literature review suggesting that 
any of the widely used tests are perfectly correlated with innate language 
ability. Therefore, to the extent the tests are unrelated to innate language 
ability, efforts to control statistically for IQ and pretest will fail. 



16 



CHAPTER 2 



STUDIES APPLICABLE TO THE ISSUE 



After gathering the studies of bilingual programs, -we read and analyzed 
each study to see if there were reasons why the study was not applicable to 
the issue 'of interest. Chapter 3 discusses those studies we found not appli- 
cable and explains our reasons for rejecting them. This chapter discusses 
the 28 studies we found ^applicable. 

The studies we found applicable exhibit the following general 
characteristics : 

o Jive studies were true experiments with random assignment (Plante, 
1976; Pena-Hughes and Solis, 1980, 1981; Covey, 1973; Kaufman, 
1968; Huzar, 1973) and one study used random assignment in three 
of four schools studied (Lum, 1971)". 

o In studies with nonrandom assignment, something was done—generally 
analysis of co variance—Co adjust for possible differences between 
.groups, either statistically or by matching. 

o Two studies seem to have overcome most of the problems of nonrandom 
assignment by very*, thorough and .clever study designs (McConnell, 
1980a, 1980b; Lambert and Tucker, 1972). 

o By definition, none of the studies relied exclusively on grade- 
equivalent scores or on the norm-referenced design. 

These 28 studies have reported program outcomes about which we can be 
reasonably confident. However, our interpretation may differ from the 
conclusion presented by the authors. Consider, for, example, the case of 
Baiasubramonian et al. (1973). These authors concluded that bilingual edu- 
cation w?,s very successful, because, in comparison to students in the English 
as a second language program, students in the transitional bilingual program 
did not perform worse in English (L2), while they were able to to improve 
their skills in their first language (LI). According to our criteria speci- 
fied in chapter 1, however, this study failed to .demonstrate the effective- 
ness of transitional bilingual education, since neither English performance 
nor performance in noolanguage subjects was found to be superior in the 
students in the English as a second language program. 

Social science researches rarely, if ever, completely free of methodo- 
logical problems. Those studies that we accepted as meeting our minimal 
methodological criteria are not without problems. The discussion of the studies* 
problems will therefore establish the upper limits of confidence we can place 
in the authors* conclusions. 

Table 2-1 s umma rizes the important /Characteristics of the various types 
of instructional programs used to meet the special needs of language-minority 
students. The third column, special curriculum, refers to whether the cur- 
riculum is organized differently from the way the curriculum is organized 

i 



in an ordinary monolingual program. For example, TBE does aot have a 
special curriculum because it uses a regular curriculum in two languages, 
whereas the Immersion method does involve a special curriculum. 



TABLE 2-1. TYPES OF INSTRUCTIONAL PROGRAMS 

Language Language 

Spoken by Spoken by Special 

Teacher Student Curriculum 

Submersion LZ L2 ■ No 

Immersion 1*2 LI and/or L2 Yes 

Transitional bilingual education. UL gradually UL gradually 

replaced by replaced by 

L2 L2 No 

English as a second language L2 L2 Yes 



UL - the child's first or home language.- 

L2 * the child's second language, the language of the school. 

' The rest of this chapter will discuss the 28 studies we accepted. 
Studies of TBE will be discussed first, followed by immersion studies. 
ESL is discussed in. studies when -it is compared to either transitional 
bilingual education or immersion. For each study, we will provide a de- 
scription, point out the' methodological strengths, and then discuss the 
findings. 
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TRANSITIONAL BILINGUAL EDUCATION 
(TBE) 
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Nam* or Study 



Author and Date 



Location 



Treatment Group* 



Comparison/Control Group 



Duration 



An Analytical Study of Secondary Freshman 

Bilingual Education and Its Effect on Academic 
Achievement and Attitude or Mexican-Americans 

Covey (1973, 1981) 

Phoenix, Arizona 

100 Mexican- Americans in bilingual education 
program 

100 Mexican-Americans in regular English 
curriculum 

9 months 

Ages : Grade 9 

Type of Program : Transitional bilingual education 

Description 

Covey's (1973) study of a bilingual program at the largest high school in 
Phoenix is important for two reasons: First, it. is one of xlie few studies 
of program effectiveness at the secondary level. Second, it is one of only 
six studies using random assignment. Unfortunately, Covey's study is very 
uninf ormative as to the nature of the program. The school year 1970-71 was 
the initial year of the ninth-grade bilingual program, bur no further details 
were given as to program structure. Mexican-American students qualified for 
the sample if they met at least one of the following criteria: (1) they 
demonstrated a limited ability to speak English, (2) they came from a bilin- 
gual home environment, (3) they manifested a reading deficiency, or (4) they 
showed a deficiency in English and mathematics. 

The school randomly divided 200 eligible students between the program and 
regular classes. The tests used were carefully selected and the following 
results were obtained at the end of the year: 

w o On the Iowa Test of Educational Development, students in the pro- 
gram outperformed the control group on the subtest "Correctness 
and Appropriateness of Expression." Ttfere was no difference in 
the performance of the two groups on "Ability to Do Quantitative 
Thinking" (math). 

o The experimental group scored higher on the Stanford Diagnostic 
Reading Test. 



* These brief descriptors sometimes oversimplify complex design features 
and appear here to give the reader a rough idea of the scope of each 
study. 
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Strengchs 

The study used randota assignment to treatment and control conditions. A 
fairly large sample size was involved with a variety of outcomes being 
measured. 

Discussion 

Although Covey demonstrates a statistically significant improvement^ in 
test scores, the program was not universally successful. Only 10 to 15 
percent of the program participants achieved a sufficiently high level 
of English proficiency to be mains treamed (Covey, 1981). 

Problems in interpreting Covey', -asults stem from the nature of the 
program. Apparently, the Phoenix ; ogram was an individualized diagnostic/ 
prescriptive program. Participants spent 2-1/2 hour* a day on reading, 
math and English. Spanish was spoken as needed. Including aides, the 
pupil -instructor ratio was about 8 to 1. Very high parent participation 
was attained with considerable use of volunteer parents in the classroom* 
Thus, when parent volunteers are considered, the pupil-teacher ratio was 
even lower (Covey, 1981). 

Hence, there are three competing explanations as to why the program 
worked: (1) individualized instruction, (2) low pupil-teacner ratio, and 
(3) bilingual instruction. 



Kama of Study 



Effectiveness of Individualized Bilingual 
Instruction for Migrant Students 



Author and Date 



McConnell (1980a, 1980b) 



Location 



Washington State and Texas 



Treatment Group 



£30 migrant children with Spanish as the 
primary language in a regular English 
curriculum 



Comparison/ Control Group 



390 migrant children with Spanish as the 
primary language 



Ages 



5-9 years old (grades K-3) 



Type of Program 



Transitional bilingual education 



Description 



McCottnell has written two reports examining an individualized bilingual 
instructional (IBI) program for migrants. The 131 program for Spanish- 
speaking migrant children had base schools at both the winter home in 
Texas and the summer work location in Washington State. The Texas site 
was located in a community where there was an emphasis on preserving the 
Spanish- heritage while the Washington community placed more emphasis on 
development of English skills. Students in the community emphasizing 
English did better on English measures while students from the community 
emphasizing Spanish did better in Spanish. Some of the program teachers 
went along on the migration to provide educational continuity between 
the two home camps. Schooling was available 12 months a year. A program 
of* individualized instruction was used for math, reading, English, and 
Spanish. The program was for preschool through grade 3. 

The method of analysis differs considerably between the two reports. 
In McConnell (1980a) the comparison group was formed by using the program 
students' test scores at entry into the program as age-adjusted pretests. 
Students of varying ages enter the program at different times and are 
pretested. By accumulating these scores over the years for each age 
level, the program generates an age specific comparison group for which 
there is little issue of selection bias since the comparison group was the 
group selected. 

McConnell's figure 4 is reproduced (see table 2-2) to help the reader 
get a better grasp of the method. Each asterisk in the figure indicates 
the results of a t-test comparing the "project norm group" to an age-specific 
posttest score. Although such a use of multiple t-tests is not the best- 
way to analyze the data, it is obvious from the figure that the results 
are robust and tnere is little use to worry over this point. 
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TABLE 2-2. McCCNNELL FIGURE 4 



2QQ+ day* 
actendanc« 



Bilingual 

IOO days 
'attendance 



Project 
Norm Croup 



ENGLISH VOCABULARY 3CORIS 
for children whose primary lan- 
guage* 1$ Spanish 




•The superiority of this scon over that of tht project nora group of the 
sas* a?e 1s statistically significant beyond tht .001 level. 

Detailed test score analysis 1s shown 1n Table 6 1n the Technical Appendix. 



Figure English vocabulary scores on Form A, Peabody Picture 
vocabulary test, of children whose primary language *! s 
Spanish, by age and attendance group compared to the 
project norm group. 

TO SUMMARIZE THE FINDINGS IK FIGURE 4: 

L Comparison to the average scores in English vocabulary of«the 
project norm group shows significant superiority for children 
in the ibi bilingual program at every age level. 

2. The superiority of children who attended for 200 or more 'days 
is statistically significant beyond the .001 level (e.g., the 
possibility that this much difference would occur by chance is 

LfSS THAN ONE IN LOOO), 

3. Children after 200 or more days attendance are markedly superior 
to those tested after only 100 days attendance, indicating 

that the gains in english are progressive -the longer the 
period of attendance. 



Based on this analytical model, McConnell found the following results: 

o Improved Spanish performance in boch Spanish* and English- 
dominant students. 

o Improved vocabulary (English) in Spanish-dominant children* 
o Improved math scores* * 

o Improved reading performance in Spanish-dominant children with 
200 or more days in the program* 

In a second analysis McConnell looked at the performance of only the 
children in the mobile component of tiui program. In the entire project f 
some students participated only in the Texas school, some participated 
only in the Washington school, and Some participated in both places* 
Instruction was also provided while on the road, where the project 
provided staff to accompany the migrant caravan (the mobile component)* 

Apparently the mobile component group of students was too small to 
generate enough data to be able to apply the design described above* An 
additional comparison group was formed V taking Spanish-dominant migrant 
students from a neighboring school, grades K-3* The program students did _ 
significantly better in English vocabulary, math, and reading (English). 
The more time participants spent in the program the better they did in 
these three areas, also. 

Strengths 

The study revealed a well-designed longitudinal analysis with a large 
sample, size and a variety of measures and comparison groups. One compari- 
son method involved using the studant as his or her own control* As childr 
of different ages entered the program, their pretest scores became the 
comparison scores at posttest for children who were younger by the pre- 
post interval* 

Discussion 

McConnell f s (1980a) study has the same problem of competing explana- 
tions as does Covey's (1973) study. There are several alternative expla- 
nations for the treatment group's improvement in performance: 

1 # Individualized Instruction . Many educators have held that indi- 
vidualized instruction is far superior to the standard classroom setting. 
The students in McConnell f s study had individualized instruction in very 
small instructional groups (with a student-instructor ratio of 10 to 1). 

2 # A Coordinated Education . A major problem in the education of mi- 
grant children is that moving from school to school disrupts education. 
Their new schools are very unlikely to be at the point in the curriculum 
where- the students left off when they left their old school. This results 
in serious gaps in education, often leaving children without the prerequi- 
sites for mastering later skills and therefore leading to very poor 

3 
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educational performance. Tht individualized bilingual Instruction program 
provided * continuoua curriculum. For tht first tiaa in their school experience, 
toss* studants wart ablt to experience tha noraal school sequence. 

3. Continuous' Education Was Available . The number of instructional days 
received was not reported. However, having services available 12 aonths ot 
the year aade it possible for children in the program to receive signiticantiy 
more days of instruction during 'each grade than the comparison students 
did. One of the besc established principles in educational research is that 
time on task (in this case, days of exposure to schooling) is related to 
achievement. 

Analysis 'of the mobile component revealed two design features that raise serious 
question about the results. First, the matching procedure was incomplete. 
Community differences were not controlled for in any of the analyses. More 
important, 121 students could have begun the program as early as age 3. 
Since no mention is made of a preschool program in the comparison community, 
the superiority of the IBI students could be attributed to the 2 to 3 additional 
years of formula instruction. 

In the second report, McConnell (1980b), the comparison group was formed by 
combining the program participants' entry scores with the scores from the 
nonequivalenf comparison group for the neighboring school. Unfortunately, 
this contamination of tha participants' entry scores with the uncontrolled 
nonrandomly selected comparison group makes any analysis using the combined 
group doubtful. 

The aethod of analysis is "o compare test scores for five categories of 
length of attendances in the lfll program (zero to 3+ years) by age- 
standardized scores. Although this results consistently favored the program 
by showing higher scores as length of attendance increased, the analysis is 
suspect for two reasons. First, an we noted above, the zero-attendance 
group was contaminated by the inclusion of a nonrandomly selected comparison 
group from a neighboring school. .Second, since each participant was tested 
twice, one in the zaro-attendance group and again in one of the. tour length- 
of-attendance categories, the scores were not independent. This is. a violation 
• of one of tbi assumptions underlying the F-test used in' the analysis. 

Although McConnell (1980b) has problems with a confounded treatment 
group, violation of the assumptions of the tests, and use or a nonrandom 
comparison group with no adjustments made for preexisting differences, w 
conclude the results presented in McConnell (1980b) can be accepted for 
the following reasons: 

o They are fully consistent with the results from McConnell 
(1980s). 

o They are internally consistent across 3 years of data. 

o They are internally consistent within the rder of the 

means for the r*"icipant only group through four categories 
' of. length-of-program exposure (excluding the contaminated 
entry scores). 
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In start, MeConnell's results show the findings are robust enough co 
hold up, even under che oethodolcgical problems introduced by che analysis 
plan in che second report (McConnell, 1980b). 
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A Study of tHe Effectiveness of the Connecti- 
cut "Pairing" Model of Bilingual -Siculcural 
Education 

Plante (1976) 



New Haven, Connecticut 



45 Spanish-dominant children 

27 Spanish-dominant children in a regular 
English curriculum 

2 years 



Grades 1 and 2 

Transitional bilingual education 



Description 

Plante (1976) reports on a,. well-designed study of New Haven's bilingual 
program. All Spanish-surnamed pupils completing kindergarten and first 
grade in an attendance area serving a large percentage of children fx'om 
low-income families were given the Inter-America Test to identify their 
language dominance. Those children in grades 1 and 2 who were identified 
as being Spanish-dominant were randomly assigned to the bilingual program 
or tb a control group. Thus, methodologically, the study was a true 
experiment. 

No bilicgual-bicultural instruction had been available to these children 
prior to the implementation ot the study. The type of bilingual instruction 
investigated involved the "pairing" model in which one native Spanish- 
speaking teacher .taught basic skills in Spanish and an English-speaking 
teacher taught speaking, reading, and writing in English. 
« 

The students were tested again after 2 years. In both grades there vr.re 
large atxi significant differences in LI reading favoring the bilingual pro- 
gram. In English, a significant difference favoring the bilingual program 
was found at grade 2 but not at grade 3 and not over both grades combined. 

Strengths 

The study used random selection and, thus, was a true experiment. Plante 
also collected data over a 2-year period. 

Discussion 

Plante was not justified in stating that "it seems significant to point 
out that the experimental pupils in the second grade and third grade, as 



11 



Name of Study . 

Author and Date 
Location 
Treatment Group 
Comparison/ Control Group 

Duration 
Ages 

Type of Program 



well as when a total group analysis is made, exc Jcf fSoring 

achievement of the control group." The only significant difference favoring 
STvas found in one grade. TBE was found no different from con regular 
curriculum in the other grade or for both grades combined. 

Plance presents results from the Metropolitan Achievement Test in grade- 
e,«IvS.«s! £ provide, no tests of the significance pf Jj^T" 
between the experimental and control groups, although all comparisons 
favored the bilingual group. 

Unfortunately, Fiance's carefully designed study is ultimately undone by 
the taiS difference between the experimental students and control group 
students £ retention rates. Ml the experimental students except one pro- 
ved through 2 years of school is the 2 years of the study Only half 
of the controls did so; the other half were retained. Thus half of the 
control g^oup had i year of school twice in the 2 years of the .cudy. The 
Sal differences in test scores between the two groups ^«"« f 6 

Sfference in failure rate was due to the fact that bilingual and regular 
cllssroom teachers applied very different promotion criteria to their re- 
C £l ^stu^ts" ^Tne expSHLncal and control groups clearly were created 
very differently in ways that did not relate to language treatmenc. Conse 
queltly, £ conclude that Plante's study suggests a program er feet but. 
ultimately fails to prove that the language treatment was eftective. 
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Nam. of Study : The Effect* of Program Models on Language 
*• Acquisition by Spanish Speaking Children 

Author and Date : Legarreta U979) 

Location : A Urge West Coast city never specified 

Treatment Group ' : 80 mono Ungual Spanish-speaking children 

Comparison/Control Group : Subsets of the treatment groups were compared 

to one another 

Duration 6 months 

Ag e3 Kindergarten 

Type of Program : 1 Submersion, English as a second language, 
transitional bilingual education 

Description 

Legarreta (1979) tested kindergarten children drawn from existing pro- 
grams. Teacher judgment, a language use questionnaire, and pretest scores _ 
were used to limit the study to students who were "essentially monolingual. » 
Five instructional models were compared: (1) submersion, (2) ESL, £J) 
bilingual— concurrent translation, no ESL; (4) bilingual— balanced bilingual 
education, no ESL, and (S) bilingual— concurrent translation with ESL. The 
bilingual programs using concurrent translation were programs in which 
English-taught subject-matter was immediately translated into Spanish. The 
bilingual program referred to by Legarreta as a "balanced bilingual pro- 
gram, using SO percent Spanish* and SO percent English, can also be charac- 
terized as an alternate immersion program. 

Legarreta found bilingual programs superior to traditional submersion, 
and bilingual education -ograms with ESL superior to programs without E5L. 
Since one traditional pre - am had ESL and two of the three bilingual pro- 
grams were without ESL, it is impossible to draw any conclusions from 
Legarreta's data as to the relative importance of ESL and bilingual 
struction. These effects are never separated. 

However, Legarreta found that children in the balanced bilingual educa- 
tion class, who received less English instruction than the other two bilin- 
gual education classes, outperformed those groups on two measures of English 
skills. The unbalanced treatments received English instruction approximately 
72 percent of the time. 

Strengths 

Leaarreta's analysis Is complex. Students were not randomly placed into 
the fiva types of treatment. However, analysis of covariance was used to 
adjust for preexisting differences. The study was restricted to monolingual, 
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Spanish-speaking children, the type of children pertinent to transitional 
bilingual education theory^ 



Discussion 



ERIC 



Legarreca found chac che more inscruccion given in the native language, 

* che baccer che performance in LI. 3uc che next finding is noc so obvious. 
The balanced bilingual* in che alternate immersion classroom were also 
found co oucperform che unbalanced bilingual groups on two measures of 
English skills. In che case of Spanish, more inscruccion led co beccer 
performance. In che case of English, more inscruccion led co poorer 
performance. 

Ac firsc glance, che language facilicacion effecc appears co provide a 
possible' explanacion for che resulc chac more iascruction in L2 led co 
poorer 12 performance.*- The language cransference cheory is generally ap- 
plied co reading, however, and Legarreca* s subjects were all kindergarten 
) children cesced on oral language proficiency. The liceracure seems co 

differenciace oral and reading language skills. What applies co one does 
noc necessarily apply co che other. Furchermore, co argue chat che crans- 
ference hypochesis applies co Legarreca' s daca concradicts ocher escablished 
principles' of oral second-language learning. Oral L2 skills are besc leaped 
by young (ages 6 co 7 or younger) children—precisely che ages or Legar- 
reca's students— and L2 mascery in young children is a direcc funccion of 
practice. ' 

There are alcernacive explanacions for Legarreca' s resulcs. The seven 
scudencs (one class) in che "balanced bilingual" program who oucperformed 
scudencs who spenc more of cheir day in English could have had an unusually 
effeccive ceacher. Such a small sample is sensicive co ceacher difrerences. 
Exceptional ceachers will gee excepcional resulcs in eicher language. 

• Another alcernacive erplanacion may be chac Legarreca did noc cesc che 
program ef feces she chinks she cesced. Two of che chree bilingual groups 
employed "concurrenc cranslacion"— chac is, che ceacher would scace a con- 
cepc in one language and chen immediacely rescace ic in che ocher language. 
Legarreca poincs ouc chac chis mechod may be less successful Chan alcernace 
immersion (the method used with the balanced bilingual group) because stu- 
dents tune out the English, knowing it will be followed by a Spanish version 
which they understand. 

Berke (1980), de Kanter (1980), and Cummins (1981) suggest that con- 
current translation is not an effective instructional strategy. Berke 
argues that linguistic theory clearly implies that concurrent translacion 
leads co linguistic confusion. The most effective learning of a second 
language occurs when the setting in which the second language is used is 
clearly differentiated from that in which the first language is spoken, 

* Briefly, the facilitation effect, or language transference theory, pro- 
poses that there are formal reading skills which transfer from one 
language to another. Therefore, it is advisable to teach reading first 
in LI and to rely on transference to help with mascery of L2. 
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u*en co the point of having different teachers' speak the two languages. 
Therefore, Legarreta' s concurrent translation (unbalanced bilingual) group 
may have performed more poorly thsu did the balanced bilinguals because of 
linguistic confusion inherent in the concurrent translation method. 

If Berke's, de Kantar's, and Cummins' arguments are correct, the 
implication of the study is not, clear . The difference between the concurrent 
translation and alternate immersion groups is exactly the same as the 
difference between unbalanced and balanced bilingual instruction.* Legarreta 
interprets the results as a test of the latter, but linguistic theory 
suggests than the former interpretation may be n»re appropriate* In 
either case, the interpretations of Legarreta's results are not clear* 
Her conclusion that balanced bilingual education "clearly is most facilitative 
of acquisition of English as well as maintenance of Spanish'* is questionable . 

We have discussed Legarreta's study at some length for several reasons . 
As one of the best statistical analyses in the literature, it is an im- 
portant study.- However, the study also shows that good statistics are not 
enough. Numbers must be interpreted. In a field as complex and as fluid 
as bilingual education, interpretation is difficult. 



Of Legarreta 's four significant tests, three involve comparing one or 
both of the concurrent translation programs with the alternate immersion 
program. This practice makes it impossible to say what her tests 
measured. For example, two of the tests pit balanced bilingual instruc- 
tion against unbalanced bilingual instruction. 
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Hang of Study 
Author and Date 
Location 
Treatment Group 

Comparison/Control Group 
"Duration 
Ages 

Type of Program 



Bilingual Education Program 

AIR (1975b) 

Cdrpus Christi, Texas 

269 Title VII bilingual education 
students 

124 non-Title VII students 

9 months 

Grades K and 1 

Transitional bilingual education 



Description 

"Although the report discusses secondary data from 4 years, apparently 
only in 1 year for grades K and 1 was any adjustment made for preexisting 
differences between the two group's. The authors refer to adjusted posttest 
means in their tables, although the text gives no discussion of how the 
adjustment was made. In kindergarten there were oo differences between 
the treatment and control groups on five English tests. In the first _ 
grade, performance by the TBE program students was superior to that or the 
comparison group on nwo Spanish tests and on two English tests. Scores on 
two more English teats were reported- to be significantly higher for the 
program students, but these results, were not adjusted and were given in 
gr ade— equivalent s . 

In addition, first- and second-grade students were categorized by the 
number of prior years they had spent in the bilingual program. A sigaifi- 
cant (analysis unspecified) result showing increasing ^improvement in English 
performance with increasing length of time in the program was reported. 
Although the analyses described- in the preceding paragraph were also con- 
• ducted for a second cohort (without adjustment), the table showing increased 
effects over time was not replicated. 

Strengths 

The Corpus Christi sample was moderately large. The authors made an adjust- 
ment for possible preexisting differences resulting f rom nonrandom select-on, 
presumably by analysis of covariance. They also made use of more than one 
outcome measure* 

Discussion 

A complete assessment of the generalizability of the study is made diffi- 
cult by the lack of detail provided in AIR 'a (1975b) secondhand account. 
Apparently the student population of the program schools is largely Hispanic, 
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for the authors note that no screening for eligibilit7 was done. All par- 
ticipants were also volunteered by parents, a situation that cannot be 
fully compensated for by the statistical adjustment. 

Generalizability is also United by the study's coverage of only two 
grades (K and 1) in one school district. 



Will Instruction in Reading Spanish Affect 
Ability in Reading English? * 

Kaufman (1968) 



New York City 



School A: 48 Spanish-speaking pupils receiving 
4 sessions of Spanish a week; school B: 27 
Spanish-speaking pupils receiving 3 sessions 
of Spanish a week 

School A: 37 Spanish-speaking pupils in regular 
English curriculum; school 3: 27 Spanish- 
speaking pupils in regular English curriculum 

School A: 2 years; school B: 1 year 



Junior high 

Transitional bilingual education 



Description 

Kaufman (1968) examined the effect of bilingual education in Spanish and 
English in two junior high schools in New York City. Bilingual education 
at this time was just beginning to receive national attention. Spanish 
instruction was given to the experimental group four times a week for 45 
minutes per session at school A and three times a week for *5 minutes per 
session at school B. Experimental and control groups received equivalent 
instruction in English. There were no significant differ' ^ices between the 
experimental and control groups in school A, In school B, two oi; six tests 
significantly favored the bilingual group, leading Kaufman to conclude, 
. "These findings suggest that there was some evidence of positive transfer 
of learning from instruction in reading Spanish to reading ability in 
English at school B." Altogether, Kaufman found a significant difference 
on two of nine tests he performed. 

Strengths 

Kaufman used random assignment in his study making it one of the six 
true experiments in the literature. He also used analysis of covariance 
to control for preexisting differences in verbal IQ, nonverbal IQ, age, 
and English pretest score. 

Discussion 

By analyzing the two schools separately, Kaufman partially lost the ad- 
vantages of random assignment because school differences are confounded 
with Che treatment. If the bilingual students at school B had an unusually 
effective teacher, Kaufman's results would be expected. Kaufman should 
have combined the data from the two schools in his analysis and tested for 
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Treatment Group, . 
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Dura c ion 
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Type of Program 
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school and teacher effects. We can only approximate such tests using 
data from the article. In school B, the difference between the bilingual 
and control means wms 0.39. That difference is reduced to 0.18 for the 
combined weighted means of both schools A and B. We cannoC say whether 
this lower difference is significant because saapla size is increased. The 
large reduction in the mean suggests that Che difference Kaufman found 
between the bilingual and control groups in school B was due to .something 
other than the effect of the program. On the other hand, the school B 
results could be indicative of the program effect while school A results 
could be due to an unusually poor teacher. The point is that something 
other than a program effect was at work, so caution must be exercised in 
drawing conclusions from the study. 

Kaufman had longitudinal data measuring word meaning ✓ paragraph meaning, 
and" total score means for English reading. Since students in school A were 
postteated three times and students in school B were posttestad twice for 
thcs£ 3 measures, Kaufman could have examined the hypothesis that, if the 
program h:d an effect, the difference between the bilingral and control 
groups would increase over time. We have computed these differences for 
Kaufman's three insasures: 



TABLE 2-3. DIFFERENCES IN TEST SCORE MEANS (KAUFMAN, 1968) 



Test 
Point 


Word 
Meaning 


School A 
Paragraph 
Meaning 


Total 


School B 
Word Paragraph 
Meaning Meaning 


Total 


A 

N 


B 


2 
3 


-.0195 
.1143 
.2152 


.1169 
.0328 
.1128 


.903 

.0747 

.1542 


.3933 .2772 
.0290 .3032 
NA MA 


.3422 
.1430 
MA. 


78 
72 
50 


49 
45 
HA 



There is no improvement over* time for school A when the bilingual and 
control groups are compared. School B shows improvement in the bilingual 
class on the first posttest for word meaning. The total mean shows improvement 
but is really again measuring- the effect of word meaning scores. The second 1 
posttest showed no significant improvement. 

. Interestingly, school A had the most Spanish instruction and showed no 
English reading improvement. However, school 3 had one period less 
Spanish instruction and showed significant gains. This, again, suggests 
that southing other than bilingual instruction accounted for the achievement 
differences in school B. 
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gang of Study 

Author and Date 
Location 

Treatment Group 

Comparison/ Control Group 

Duration 
Ages 

Type of Program 



: An Evaluation of the Effectiveness of 

Selected Experimental Bilingual Education 
Programs in Connecticut 

: Zirkel (1972) 

: Connecticut (Bridgeport, Hartford, New Britain, 
New London) 

: Grade 1: 73 Puerto Rican children; grades 
2 and 3; 53 Puerto Rican children 

: Grade 1; 73 Puerto Rican children in a regular 
English curriculum; grades 2 and 3; 76 Puerto, 
Rican children in a regular English curriculum 
with ESL 

: 1 year 

: Grades 1-3 

: Transitional- bilingual education 



Description 

Zirkel (1972) looked at three variations of bilingual programs in four 
towns. The author points out that in some cases what was termed a "bilin- 
gual class in one location was indistinguishable from what had been indi- 
cated as a -control- class in another area. The author characterized these 
variations as (1) the bilingual education model and (2) the quasi-bilingual 
model. The children involved in the study were "economically disadvantaged" 
Puerto Ricans. The use of LI in the classroom varied considerably in the 
schools included in the study, ranging from 10 to L50f minutes daily. 

r 

Adjusted posttest means in both English and Spanish reading were analyzed 
by five tests. There was only one significant difference favoring the 
bilingual program in English (and one in Spanish for the same group of " 
students). This difference occurred in grades 2 and 3, which had rhe most 
English instruction. The .analysis of quasi-bilingual models disclosed 
nonsignificant differences between the experimental and control groups in 
3-hieveaent outcomes. 

Strengths 

To partially control for the effects of nonrandom selection, Zirkel 
matched the experimental and comparison groups for age, sex, and socio- 
economic status by eliminating students from the study until group level 
matching was attained. Analysis of covariance controlled for pretest 
and nonverbal IQ. 
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Discussion 

Bilingual teachers in grades 2 and 3 were more qualified than first- 
grade teachers. Therefore, teacher differences probably had a significant 
impact on the higher level of student achievement in grades 2 and 3. 
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gage of S tudy : An Evaluation of Title VII Bilingual/Bicultural 

Program, 1977-1978 School Tear, Final Report 

Authors and Date : Ames and Bicks (1978) 

Location : Brooklyn, New York 

Treatment Group : 212 Spanish-speaking and Creole-French- 

a peaking children in bilingual and ESL classes 

Comparison/Control Group : 457 Spanish-speaking and Creole-French- 

speakiag children in regular English curriculum 
with % soma ESL pullout 

Duration : 9 months 

Ages : Grades 1-9 

Type of ProgTam : Transitional bilingual education, English as a 

second language 

Description 

Ames and Bicks (1978) present an interesting study of a bilingual program 
in New York City. Participants in the program were Spanish speakers and 
Creole French speakers who scored below the 20th percentile on /.he English 
Language Assessment Battnry developed by the New York City Board of Educa- 
tion. The comparison grtxip was composed of students with non-Hug lis It- 
language backgrounds in schools wher^the bilingual population was not 
large enough to justify a full bilingual program, but where the s ame 
English-language instruction was given on a pullout basi3 as was done in 
the schools with bilingual programs. The bilingual and pullout groups of 
students received 3 to 5 hours of English instruction a week. It anpears 
the ESL group had more English instruction than the other two groups. 

■ The size of the bilingual and ESL classes was less than 25 students. 
All teachers were fluent bilinguals. Some of the teachtrs used individu- 
alized instruction, while others chose -traditional" styles of teaching. 
A comparison of the bilingual versus ESL versus pullout groups in English 
reading showed significance. Although the bilingual and pullout groups 
had equal gains, the ESL group had the largest gc «,n. 

The authors conclude: 

Since the time spent in intensive study in English was 
Che same in both the bilingual and pullout groups, it was 
understandable that achievement in reading English was 
not significantly different. In math, however, •• .the 
results. . •indicated that those students who received 
instruction in the native language achieved higher scores 
in math than those whos^ instruction was given in English. 
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Strengths 

Analysis of c ©variance was usad by Ames and Bicks to adjust for pre- 
existing differences due to nonrandom assignment. The method of selecting 
the coapa risen group was very well devised and inspires aore confidence in 
the equivalence of the treatment an d control groups than is usually the 
case. 

Discussion 

' The ESL group is never full described in the report. We are never sure 
how aruch English instruction they received. However, on the reading test 
tha ESL students showed gains 50 percent higher than either the bilingual 
or pull out students. The authors present an insignificant F-test for bi- 
lingual versus pullout, but, unfortunately, no separate test of SSL versus 
either of the other two program types was given. In aath the pullout 
group gained 49 percent of the bilingual group's gain, but the ESL group 
achieved 80 percent of tbe bilingual group's gain. Again, no paired tests 
involving ESL ware given- Because no tests of ESL other than the overall 
F-test were given, rod because we are unsure whether the authors used 
adjusted or raw aeaas, it is unclear whether the ESL program nay have b-en 
equal or superior to eh& bilingual program. 

Another problem with the analysis is that the U..ingual versus pullout 
test was apparently a two-variable analysis of. covariance done after the 
three-variable test. This is not the preferred way of testing differences 
between pairs of aeans after ffnding a significant F-test in the omnibus 



test 



The study may also be confounded by teacher differences, as some teachers 
employed individualized instruction and others employed traditional methods. 
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Name of Study 
Authors and Dace 
Location 
Treatment Grouo 



Iloilo I and Rizal 

Ramns, Aguilar, and Sibayan (1967) 

The Philippines 

Iloilo: 232 students instructed in Hiligaynon 
and then English; Rizal: data unavailable 



Conparison/Control Grouo 



Duration 



Ages 

Type of Program 



: Iloilo: 301 students instructed in English; 
Rizal : data unavailable 

: Iloilo: 6 years; Rlzal: 5 years 

: Iloilo: Grades 1-6; Rlzal: Grades 1-5 

: Transitional bilingual education/ immersion 



Engle (1975) thoroughly described a 20-year study of instruction in the 
first language in the Philippines, which we quote here. We have divided 
her description into sections for clarity. 

Iloilo I and Rizal (Ramos et al. , 1967) . The Philippine 
studies referred to by these place names represent two 
carefully designed studies executed between 1948 and 1967. 
In some respects, the results conflict with each other, 
and one can begin to understand come of the factors only 
by examining the differences between the two [and] then inte- 
grating the findings. The language situation in the 
Philippines is complex; there are many vernacular lan- 
guages. Tagalog (or Filipino) is the national language, 
and the government would like to encourage English in the 
elementary schools as an additional national language. 

Description : (Iloilo I ) 

The original Iloilo experiment was conducted in an area of 
the Philippines that speaks Hiligaynon as its mother tongue. 
The Iloilo I study was designed to analyze the ef rects of 
initial instruction in the vernacular on the eventual 
learning of the curriculum in English. 

The study was statistically well designed. The experi- ■ 
mental group received instruction in the vernacular for 
grades 1 and 2, an<^ in English in grades 3 through 6. The 
shift was abrupt. The controls were given all instruction 
in English from grades 1 through 6* Instructional mate- 
rials were identical throughout, with the execution that 
the first and second grade experimental materials were 
translated from English into kiligaynon. Assessment of 
abilities and achievement occurred before grade 1, and 
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after each grade through grade 6. Assessment in grades 1 
and 2 was in the vernacular for the experimental group, and 
in English for the controls. All of the assessment was in 
English for the 3rd through 6th grades. Tests of reading 
arithmetic problems, understanding social studies, and 
learning language skills were given at each grade. 

At the beginning of the project the sample included l,lo4 
controls and 758 experimental* ♦ When the experimental 
were matched with controls, the sample size decreased to 
188 experimental* and 189 controls; Unfortunately, the 
attrition rate was very high; by the end of the 6th grade, 
only 23 percent of the sample remained. Reports of the 
actual numbers differ. - The report of the 6th grade evaluation 
indicates that 232 E's and 301 C's remained in the study. 
Of these, 82 were matched. 

The methods employed' for second language teaching were dif- 
ferent from the standard methods. The new method was based 
on language patterning and drills, emphasizing both structure 
and sound relationships. Teachers were given training in 
teaching in Hiligaynon and in teaching English as a second 
language • 

At the end of the first year of the study, the experimental 
group was significantly higher on reading (in the * language 
of instruction) and social studies. The differences in 
arithmetic were not significant. At the end of the fourth 
year (two years of instruction in English for the experimental 
group, four for the control ) , a nonsignificant superiority 
in the control group was found for reading and arithmetic, 
and a significant superiority was demonstrated for language. 
Tha experimental group had a slight superiority in social 
studies. 



Strengths : Iloilo I 



The sanple consisted of fourteen elementary schools equated 
for SES, teacher quality, the principal's qualifications, 
and supervisors. Experimental teachers were generally of 
higher SES than controls. Children were further equated 
on the Philippines Mental Ability Test, chronological age, 
and school attendance. 



Discussion: Iloilo I 



The confusion with which the project is reported and quoted 
is exemplified in a comment by Venezky (1970) who reviewed 
the study. He reports that an independent investigation 
of the fourth-year results by the Director of Public Schools 
showed significant superior performance by the controls on 
all tests, including social studies. Venezky includes no 
references. He alludes to the "overenthusiasm" of the 
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program's director for the native language approach as a con- 
founding factor, and this observation of enthusiasm is echoed 
in the report by Ramos et al. (1967). Although one cannot 
discard a study because the director believes in it, one can 
suggest that the Hawthorne effect may be operating in a situa- 
tion where one group is seen as more exciting and more signifi- 
cant than the other. No control group matched on* "being studied" 
was included, nor did the investigators seem aware of this 
problem* 

In the sixth year evaluation (with the reduced sample size), 
the experimental were superior to the controls in social 
studies achievement tests and slightly higher in arithmetic 
and reading tests.. The controls scored slightly higher in 
language . 

A Personality Inventory was given at the end of grades 4,5, 
and 6. Children in the experimental groups reported themselves 
significantly higher on one of the 4 or 5 dimensions of that 
test, though the dimension varied from year to year. 

The results of this study were widely accepted in the Philippines; 
all children were then* given instructions in their vernacular for 
the first two years of school, and in English for the remainder. 
We feel that such an adoption of policy was premature, particu- 
larly when the sample was so small, and when only one model had 
been attempted. Other variations, such as introducing the second 
language in the first grade as a language of instruction, were 
not mentioned. 

The study suffers from a number of problems typical of many such 
studies. The tests were inadequately validated in English and 
then simply translated into the vernacular. Variables were not 
isolated; a new method of instruction was confounded with the 
basic hypothesis (differences in language of instruction). No 
control for the Hawthorne effect was made; the children could 
well have been achieving because they felt special. The two 
curricula in first and second grades were unequal since the 
English materials were published and polished, whereas the 
Hiligaynon lessons were on "rough dittos, often unclear." The 
level of knowledge of English on the part of the teachers is 
recognized as extremely low. The high drop-out rate suggests 
that the final sample is extremely select in terms of the 
factors which permit a child to stay in school— probably related 
to SES. 

Even though severe criticisms can be raised methodologically 
about the interpretation of the study as a test of the reading 
transfer and oode-of-instruction hypotheses, it does indicate° 
clearly that experimental children in this situation were not 
hampered in achievement. If, that is, they were able to stay 
in school for 6 years. 
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Description ; Rizal 



The Rizal study was designed co gather information as to 
the most appropriate time to introduce reading in English 
and English as a medium of instruction, questions that had 
not been answered by Hollo I. Five groups were defined 
according to the children's grade level at the introduction 
of English for reading and as a medium of instruction. • 
F. B. Davies of Hunter College, New York, served as con- 
sultant for these two studies. 

Grade in which English Grade in which English 

is first used aa a reading begins 

medium of instruction First Second 

First Group 1 

third Group 2 Group A 

Fifth Group 3 Group 5 



Teachers received instruction on the teaching of English, 
and teaching various subject matters in English. They 
received no instruction in the teaching of,' or with, the 
vernacular (in that location, Pilipino) and continued to 
use outdated material that had followed from the recommen- 
dations made after the first Iloilo experiment. 
» 

The time at which reading in English was introduced appar- 
ently made little difference on an English language reading 
• test . 

To test the hypothesis concerning the medium of instruction, 
three versions of all achievement tests were constructed: 
English, Pilipino, and bilingual. The results suggested 
that varying the medium of instruction did not hava a large 
effect on basic skills. The only effects w re on arithmetic 
and language scores. Those who had been introduced to English 
most recently scored highest on the English arithmetic test. 
Those who had used English as their medium of instruction 
the longest had the highest scores on the English language 
tests in sixth grade* 



Strengths : Rizal 



Schools were systematically selected, equated on significant 
variables, and carefully matched. 



Discussion: Rizal 



The authors argue that English competence is directly re- 
lated to number of years studying English. Cae might 
therefore argue that if you want children to learn English, 
you should begin as soon as possible with English, since 
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ic does not retard the other language and the children will 
have learned English better. These results contradict the 
findings of Iloilo I. However, on closer inspection a 
third variable might explain the discrepancy. 

As noted previously, the teachers were not trained in the 
use of the vernacular in the Rizal study; all of their 
training vas in the use of English. On the other hand , 
the majority of the training in the first study was in the 
usa of the vernacular for instruction. Thus a combination 
of teacher training effects and Hawthorne effects might ex- 
plain the difference between studies. Ramos et al. (1967) 
themselves suggest such an explanation. A future direction, 
then, could be an assessment of teachers' capabilities and 
the effects of teacher training on achievement in various 
areas. However, neither of the Philippine studies conclu- 
sively indicates whether introducing reading in the first 
language should be introduced as a language of instruction. 

We conclude, as does Engle, that these studies do not clearly show co: 
sistent superiority of either transitional bilingual education or immer- 
sion. Therefore, we classify the studies as a single study showing no 
difference between bilingual education and alternatives. 
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Name of Study 

Author and Dace 
Location 
Treatment Group 

Comparison/Control Group 

Duration 
Ages 

Type of Program 
Description 



An Investigation of the Effeccs^of-BagJtground 
Characteristics and ^SpaciralUnguage Service 
on the ReadijwAchievement and English 
Fluency -of Bilingual Schools 

Matthews (1979) 
Seattle, Washington 

383 language-minority students in bilingual 
and ESL programs 

1,011 language-minority students in a regular 
English curriculum 

9 months 

Grades 2, 4, 6, and 8 

Transitional bilingual education, 
English as a second language 



-Matthews (1979) analyzed the differences between all language-minority 
children in the Seattle special language service bilingual ?3afand 2l 
language-minority children not in the program. Special languaS service! 
included English as a second language and/or bilingual instruction!^ 

Chinese T?h r ph1l? h ^ dren , WiCh 3 n — S ^h-speaking background were 

aS Mte/lSSn? " ' K0rean ' SpaniSh) Ja P anese ' Samoan, Vietnamese, 
and other ethnic groups represented as well. 

The exact nature of the program Matthews examined is unclear, being re- 

-illuzt° f SPeCi f 1 langUage 3ervi « 3 '" «as defined as including 
SUM-* ma r. a 3 u ^^...and/ov bilingual instruction in required 
subject matter: mathematics, science, health, social studies, language 
arts. However, the program was established under a Lau compliance agreement 
calling tor T3E wherever practicable. compliance agreement 

,r jUS C $ e,i ! P " 3en " ^ on all language-minority students tested in 

' * ' f iD Che SeaCtle sch00ls in che ot 1979. The 

trom fall 1978 for grades 2, 6, and 8; the five-point Lau scale of English 

status)' "fJhtiS 8 ^ and f" 6 lmCh (a pr ° Xy family socioeconomic 
!nJ r Cheorecical mdel relating the control variables to each other 

bv oarM^r 06 " Wa3 d r el0Ped * ?" coo P° nen » of the model were tested 
by partial gamma coefficients and X 2, a procedure somewhat similar to 
log linear modeling. About one-third of all bilingual students received 
bilingual services, with 56 percent of the students scoring in thelowest 
itlltnl'V??*^ 1 ** Serve ?* Thu3 ' W««l»c«l7 equal numbers of low 
ItrZ J , We "u in Che Creacaenc ™* -acrol conditions, which is 

very desirable from che perspective of the analysis. 
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The errecc of THE on achievement was tested by holding fluency, language 
background, income, and grade constant, forming 64 contrasts. There were 
six significant coefficients, all indicating lo*er achievement on the MAT 
total reading score (English) for students in the program. 

iQ7a n /f s * cond anai y sis Matthews examined gain from fall 1978 to spring 
1979 (1 year of school) and found no difference between the prop*, -ions 
of students gaining against the norm in the treatment and control croups. 
Matthews states: 



...served students tend to score lower than not served students 
even when background variables such a fluency and previous 
achievement are held constant. In addition, the served and cot 
served students tend to improve their reading percentile scores 
in similar proportions (thus students who are not served progress 
in English reading skills as rapidly as those who are served). 

Strengths 



With a larger number of students, Matthews was able to carry out an 
analysis that has a number of advantages over the more usual approach 
using analysis of covariance. By using nonparametric statistics, the 
analysis avoids having to make a number of problematic assumptions about 
the nature of the underlying distribution and about the metric that are 
necessary in a parametric analyses. The one clear advantage parametric 
methods have, their greater power in detecting a difference, is cot 
necessarily a problem here since the use of less powerful tests makes 
the analysis more conservative. Matthews may have overlooked a small 
true effect by using less powerful tests, but if so, the magnitude of 
the effect would be so small as to have little practical valre. 

Discussion 



The generalizability of Matthews' findings is limited primarily by the 
restriction to one school district and by the relatively low proportion 
of Hispanic students in the school (only 9 percent of the language-minority 
students were Spanish speaking) , An occasional problem is that, even with 
the large number of students, some of the 64 cells in the tests table con- 
tain so few students that the X 2 test may not be accurate (a point Matthews 
recognizee). 

It is worth repeating Matthews' final statement because It reminds us 
of the limits ultimately imposed on all the nonexperimental evaluations 
discussed here: 



Given the limitations of the current data base, it is 
impossible to determine with any confidence whether 
the results are a reflection of the effects of service 
or whether they merely reflect current implementation 
practices. 
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Naat of Studp 

Author and Dace 
Location 
Treatment Group 

Comparison/Control Gmnn 

Duration 
Ages 

Type of Prog ram 
Description 



: Aa Evaluation of Seme Cognitive and 

R^ CtiV ? ***** ° f a S P ani sh-EngUsh 
Bilingual Education Program 

: Skoczylas (1972) 

: A community in northern California 

: 25 Title VII students (Anglos and Mexican- 
Americans) 

: 22 ^ l0 K Mexican -^ricans in a reguia 
English curriculum 

9 months 

Grade 1 

Transitional bilingual education 



Skoczylas (1972) reports on a Title VTT mh m , 
in first grade. The bilingual educatio ^class X^" 8 *" f " childrM 
came from two different schools in I d Che con P ar ison class 

Each class contained native solaLrs of 1^ ** mbmn Califo " 

zssn a s rrS- tJ?-" p"- s sf:;: e 

* option of enrolL^t^ir^n 1 St™ oTan^eT *^ 

and two aides, all three blUn^^.i^ci'SSS:" ^ ""^ 

^X^r!^ - see how 

ables of age, I Qj h£e^ c SS™i J \ ^ differed on the relevant vari 
educational background iTnlltl* a ™ ir0Tm ™> "hool attendance, oaren: = 

variables were used" as covarianc ^art^aXS S?"' ^ taCkgr0UBd 
of che best we encountered for tn^reSf varLo^s . 

wSTSo^l^J" C ° performance. 

was worse"! Skocy!.* study S"lS3 to^e ^ ° aCh P" f ormance 

generalizations to all gradL are n« jus d " 
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Strengths 



The author collected extensive background information from the oarents 
tested for initial equivalence of the groups, and used analysis of'covar-' 
iance to adjust for initial differences. 



Discussion 



Skoczylas (1972) found no difference be We en program and comparison 
fi.sc-grade students in Spanish listening comprehension and English skills. 
The program students performed significantly better in Spanish-speakinx 
skills and significantly worse in math. ^ 

ing^rcrTtlcis^; 3 (198U eValUaCi ° n ° f ^ S,,t0C2ylaS 3Cudy ' hi » hli ^ hc - 

Skoczylas' listening comprehension test (one of two tests of English 
used) consisted of 11 yes/no responses to quesfins about a brief para- 
graph. Thus a child could be expected to get 4 to 7 correct responses by 
chance. On the Spanish version of this comprehension test, the mean pre- 
test score for the control group was 2.2, well below what would be pre- 
dicted even by chance response to the test. 3y posttest time at the end 
or the year, this raw mean climbed to 4.8, essentially moving into the 
range of chance response. (For the experimental group, raw group mean ' 
scores were 6.0 and 8.0 for P re- and posttest.) The experimental £"p 
was rated 48 percent Spanish-dominant or Spanish-monolingual; 23 percent 

: u u er V f Che concro1 8T«»P were in these categories. This, along 
with the home language usage data, seriously calls in question just how 
bilingual the children in the control group were. The analysis of co- 
variance did not adjust for language use in the home. 

The children in the experimental program were superior in the pretest 
to the children in the control group on three of the four measures of 
language use including both measures of Spanish use and one measure of 
cnglish jse (the other English measure showed no significant difference), 
cms difference in groups is important when one considers that the child- 
ren in the experimental group had gone through a year of bilingual in- 
struction in kindergarten, while the children in the other treatment had 
gone through regular kindergarten. This pretest is in some sense a com- 
mentary on the effects of the bilingual kindergarten program. It is in 
large part the superior performance of the children who had been in the 
Che study ki2d<irgarCen chaC forces che use ° f analysis of covariance in 

All the children in both groups took the math test in English. Half 
the children in the experimental bilingual program had had math instruction 
only in Spanish, with no prior math instruction in English. Thus, these 
children may have encountered the technical terminology of math concepts 
in English for the first time on the posttest. 

There was no pretest in math, only a posttest. Nonetheless, the analy- 
sis of covariance had the effect of depressing the math scores of the 
cnudren In the experimental bilingual program snd increasing the scores 
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of the children in the control group, because it was compensating for the 
superior language performance of the experimental children on the three 
language neasures in the pretest. There are insufficient data oresented 
to determine ir the groups would continue co be significantly different 
if actual scores, rather than adjusted scores, had been used. 
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Nane or Study 2 ^ Effectiveness Study of English as a 

Second Language (ESL) and Chinese 
Bilingual Methods 

Author and Date 2 Lum (1971) 

Location : San Francisco, California 

Treatment Group . 35 n»no lingual Chinese students in a 

bilingual class 

Compari3on/Concrol Group : 20 monolingual Chinese students in an 

ESL class 



Duration 



9 months 



A g es . : Grade 1 

Type of Program s Transitional bilingual education, English 

as a second language 

Description 

Lum (1971) compared a program of English as a second language with 
transitional bilingual education. The students were 55 monolingual 
Chinese-speaking first graders in San Francisco. Chinese language arts 
were caught in the TBE schools but not « the ESL schools. Sthough" 
little Chinese was used in the ESL program for content-area instruction 
^lu?^ W 3 P° kan u in 1 che ^ E schools. U3e of Chinese in both schools 
uacliced over the school year, becoming very minimal by the end of *he *SL 
scnooi year. English language arts instruction was given through ESL pro- 
cedures in both schools, but the ESL-only schools averaged 50 percent 
more time each day in ESL instruction. . P eccenc 

All the teachers involved in the study were Chinese. The biliazual 
• ceases were team taught, so that one of the two teachers was expelt in 
Chinese. The ESL classes had only one teacher per classroom. 

«™nifT nC ?« Che creacaenC3 "ther complex. First, students were 
screened by self-report, teacher judgments, and the Hoffman Bilingual Scale 
to identify only monolingual Chinese-speaking students for the study. Stu- 
dents who lived in one area were randomly assigned to one ESL and wo T3E 
schools. Students living in a second attendance area were all assigned to 

Ind SJ??S" 1 The " W6 ? e 35 biliR « ual and 20 SSL students, with wo ESL 
rh , 5" !, cla33es - reviewicg.the assignment process Lum concluded 
that subjects seemed matched through pretesting and randomization by area 
or residence. 7 

Lum measured oral proficiency in EngUsh using ratings of tape-recorded 
responses to teacher-administered stimulus pictures. Apparently the 
pictures and rating method were taken from standard procedures described 
in the literature, although Lum's wording implies they were modified. 
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Only one person did the scoring of the capes. In general, free-response 
scoring systems are best done by Ruleiple racers in order to achievT ac- 
ceptable levels of reliability. 

The students' responses to the pictures were rated on five scales- 
length ^ response, length of the five longest responses, number of dif- 
ferent words used, structural complexity, and grammar. There were no 
differences between the groups on the last two items. On the first three 
measures the Fnglish as a second language students significantly outper- 
formed the transitional bilingual education students. Lum converted the 
scores to age norms and found that the TBE students were, in the first 
grade, functioning in English at a level equivalent to native English- 
speaking o- and 4-year-olds. ESL students were performing at a level 
equivalent to native 3.6- to 4.6-year-olds. 

Lum reported that, in the free-response situation, students tended to 
*eply in the language most used by the teacher. That is, the more English 
was use* in che classroom, the more English the students used in their re- 
plies. No data or analysis on this point are provided, however. 

Lum looked at differences between classes within each instructional 
method on each of the five measures. There were no significant diffe- 
rences among the three TBE classes on any of the five variables. There 
was one significant difference between the two ESL classes on one of the 
tive variables. In general, then, chere seems to be little problem with 
Lum s data due to either nonrandom assignment selection bias or teacher 
effects (except in 1 out of 10 comparisons). Nevertheless, it would have 

f 16 ? Che author had Cakan toore e*t««*ive steps co introduce 
additional statistical controls for these effects. 

che L ^r °S Ca ^ ed 3t !? nt « lf -"P°"s of use of first language outside 
the school. Englisn performance was negatively related to use of LI out- 
side the school and use of LI outside the school was positively related to 
being in a transitional bilingual education class. 

Strengths 

The project was restricted to monolingual Chinese speakers. There was 
a comparison of two types of instruction. Since most of the students were 
randomly assigned, che project was almost a true experiment. Extensive 
measures were used. 

Discussion 

Lum's categories are somewhat misleading since the SSL classes used 
Chinese in their subject areas, although to a lesser degree than the TBE 
classes. The gene ratability of Lum' 3/ findings is limited for several 
reasons: che absence of measures of literacy and writing, a small sample 
composed of one grade in one school district, the unknown reliability of 
the method, and the young age of the students. However, the following 
conclusions are suggested by his study: 
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o English perzoraance is a fraction of exposui . zo English, 
o English as a second language alone (with she liaised use of LI) 
"coS^ngu'age. 3 " ? lus « * 

S«J: U ?riJ! CUd ? ^ n ° C Pr " enC any daCa on Earning in nonlanguage subjects 
So« Proponents of craasicioaal bilingual education argue chat Inst rue La 

learning cf -P*r only to 
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Name of Study 



Models of Bilingual Education : Comparisons 
of Effectiveness 



Authors and Dace 
Location 
Treatment Gro up 

Comparison/Control Group 

Duration 

Ages 

Type of Program 



Moore and ?arr (197 ) 

Small > rural town in West Texas 

130 Spanish-dominant students with 
limited English proficiency 

77 English-dominant students 

9 months 

Grades K-2 

Variations of transitional bilingual 
education 



Description 

Moore and Parr (1978) studied 130 language-minority and 77 English- 
dominant students in grades K through 2 iv: four schools in a Title VII 
project. Four types of programs were represented; maintenance bilingual 
education, transitional bi Ungual education, minimal bilingual edu-ation 
Uot more than -0 minutes a day of formal Spanish instruction), and non- 
bilingual (all-English) classes • 

The school district was located in a small, rural community in West 
Texas where Spanish is the home language of roughly one-third of the 
students. Thirty percent of the students were from low-income families 
and many families were highly mobile because of seasonal work. 

Moore and Parr summarize their results: 

Non-bilingual classes scored significantly higher than 
bilingual classes on measures of reading and language 
achievement in English. Because students were not 
randomly assigned to treatment groups, these results 
should be viewed with cautiou. Covariance was used to 
attempt to correct for pretest differences, but covariance 
systematically underadjusr.s for initial differences between 
groups. 

In addition, there were co differences among the groups on the math 
scale of the CT3S. Additional analysis by the authors chat sex and 

an unspecified rating of teacher competence had significant effects on 
some of the measures. 
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Strengths 



An analysis of covariance using socioeconomic status, language domi- 
nance, teachers, teacher competence rating, aide competence, and school 
was employed. Thus, this study has better statistical control than mar/ 
studies having nonrandom assignment. 

Discussion 



There are some problems with Moore and Parr's analysis- Sex differences 
and levels of teacher competency should, have been controlled. It is sur- 
prising, however, that classes with teachers rated as les*; competent zan- 
erally had better results. 
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Same of Study . Acadiana Bilingual Bicultural Education 

Program: Interim Evaluation Report 1978-79 

Author and Date : McSpadden (1979) 

LocaCion : Lafayette, Louisiana 

T reatment Group . 142 Ticle ni 3txldenZ3 (blacks J'^, 

Comparison/Control Group : 54 students (blacks and whites) 

Nation : 9 months 

^5£1 : Grades K and 1 

Type of Program , - -^clonal bilingual education 

Description 

McSpadden (1979) reported on the first year of a French-English Title 
SoSSS" t a 3 Che projecC included ^th public and 

Sade ^°?7 ?ar - tici P a f f and con » 0 ^ i» kir^ergartek and first 
grade. About 37 percent of instruction was in French in language arrs 
math and social 3 ciences. A bilingual specialist, associatftfachers 
and aides caught che French portion of che curriculum while regular 

re^L 0O ceacM-: rS * < laSse * ***** o*L?%£g£ The 

regular teachers performed support activities during the. French instruction. 

<s Tll^ h Ch r- Sch00l / S ^ CC 3elect ch£ Participant and control groups 
4es 1 'wS" 31 ? °f covariance was used to adjust pretest Sffer- 
^ i^S ^ 7 developed French language test of French skills, math 
and social sciences, program participants had significantly greater kins 
over the school year chan did nonparcicipancs. 8 

in ?!!.* 3Candardi2ed .achievemenc «sc in English, chere was no difference 

.»b! i * rr0manc u e or che wo S^P 3 °° any of che various Ungual sS 
subscales or on che mach porcion. * s oK - xx 

Strengths 

McSpadden employed analysis of covariance to adjust for pretest di-'fe- 
ences, and his sample size was fairly Large. ? 

Discussion 

n.rS e , Pr0gram i mpr0Ved u U P erforaan « no detrimental effect on English 

2™"' °» «*« ocher hand, chere were no gains in che scudencs' Jfr- 
£S?E " a re3ulc o£ d *^y instruction in LI (i.e., no 

facilicacion effect). Therefore, since che cricerion used for che present 
paper is one of improved performance in English or ocher subje,c maccer 

2 ITtlt* ^ Spadden sCud ? found "ansicional bilingual education 

to be ineffective, subject to the following limitations: 
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Ic could cake more Chan i year for che facilitating effect 
occur. 



o There are no daca presented co show that scudencs in che progr 
were language-liaiited. Inscead of being a language-ainoricy 
group struggling co learn English, they could basically be an 
English-speaking group learning a second language (French). 

o The facilicacion effect is generally discussed in ceras of 
Uceracy, noc oral proficiency measured in young children. 
First-grade students are just beginning to learn reading and 
writing skills. The facilitation effect would not yet be 
applicable. 

o The generalizabilicy of the results is limited since the study 
deals only with kindergarten and first grade. 
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flame of Study 

Author and Dace 
Location 
Treatment Group 
Comparison/ Control Group 
Duration 

Type of Program 



Acadiana Bilingual Bicultural Education 

Program: Interim Evaluation Report 1979-60 

McSpadden (1980) 

Lafayette, Louisiana 

203 Title VII students (black and white) 
60 students (black and white) 
9 months 
Grades K-2 

Transitional bilingual education 



Description 

proSff?^ 1 1980) conCinued che evaluation of the Lafayette bilingual 
program into the next year of operation. One additional grade was added 
to the program, which now included kindergarten through second grade, 
Sr 3 prog P ?e r ss d in 3 u! C C ° mPare ^ CreaCacnC S*«P and comparison 8 rcu ? 

Using the same analysis as the year before (McSoadden, 1979) the studv 

c ch?££oSS sh prosress of che eomparison and ^ ecc " ud -^ *™l 

o There were no differences in kindergarten; 

o Project (transitional oilingual education) students in grade l 
had significantly lower total reading scores than che comparison 
students; and r 

o Project students in grade 2 showed significantly poorer performance 
on word knowledge and math subtests. 

Strengths 

ore^st^fS" USSd 3 ^ a r mPle SiZ *' lon S iCu di^ data, adjustment for 
pretest differences, and analysis of covariance. 

Discussion 



Although che results are subject to the same limitations discussed pre- 
TlTn S \l<°?'^°&* S **4*« reporc, chey screngchen che interpreta- 
tion chat che program is noc erfeccive in developing skills in English and 

«,,L, r 6Xten u ClUC 814(16 2 P»"icip.nts have noc been replaced bv 
student turnover che grade 2 scudencs represenc a longicudinal cohort ' 

n l J 6C J iV ! d I 5 l ° 30 per " nC ° f 311 iCs «™ h instruction over 3 years 

JS"; *J e S ugh Chese "udencs performed che same as che comparison group 

behind. 78ar * an additioa ^ y«r shows chey are beginning to fall 

41 



ttame of 



Study 



Authors and Date 
Location 
Treatment Group 

Comparison/Control Group 
Duration 



Type of Program 



: Do^Bilingual Education Programs Inhibit 

English Language Achievement? A Report on 
an Illinois Experiment 

: 3alasubran»nian, Seeley, and De Weffer (1973) 
: Illinois (excluding Chicago) 

: ^13 Spanish-speaking children in a transitional 
bilingual education program 

: 104 Spanish-speaking children in an English 
as a second language program 

* 5 months 
: Grades K-3 

: Transitional bilingual education, English 
as a second language 



Description 

^^^"T 1 ? 11 f C U973) con P a » d 213 students in a bilingual pro- 
cesses The sen 1Un0lS ? Ch 104 S ?^-*^*S children^ E^onlv 
classes. The scudencs were in kindergarten through grade 3. 

re*uiar 3 L 7 r?S d ^ n inscrucCion in SngUsh language arts as 'a 

f?«IrfS" V?' Curriculun and an additional 30 to 40 oiinutes daily 
in special English as a second language instruction. It aopears chat c-e 

and ^T^°Y am - C ° nSi3 l ed ° f ° ae " half day in che cradicW curriculum 
was -i a \^< Che ? ilin S" al P'ogTM. English as a second language 

'as part or the bilingual component. Thus, the bilingual program children 
were exposed to approximately 25 percent less English during the sSool 

S rr a l V V , CnUdr " Wh ° We " CaUghc En * lish " a «•«□,«. language in 
tha traditional curriculum, *»"s^*se j.n 



The 
groups 
cation 
temal 
a nee ve 
authors 
student 
present 



over rh. * f ^ ° f aCCriCion was «■ same for the two 

Z ItSZ ? /"I ^ d presenC a length ? discussion of the aopli- 
to their analysis of Campbell and Stanley's (1963) threats to in- 
and external validity. Mo differences in English-language p.rfo ta- 
re found between the ESL and bilingual- ? lus-ESL group!, Lading the 

to conclude that the bilingual program was a success since thl 
s improved their LI skills (although no evidence on this ooint is 
ed in Che paper) at no cost to theil English performance.' 



Strengths 



The study used a large sample size from several schools comoaring two 



types or instruction. An analysis of covariance was run to cont-ol <or 
nonrandom assignment. In addition, the authors examinS pretes *dif erences 
and round a significant pretest difference occurred in one of the three 
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Discussion 



Finally, che authors presented no inforaation on how the insertion* i 
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Name of Study 



Auchor and Da ce 
Location 
Treacmcnc Group 

Comparisca/Concrol Group 

Dura don 
Ages 

Type of Program 



: 311inguaJL Education in San Juan County, 
(Jcah: a Cross-Cultural Emphasis 

: Coccrell (1971) 

: San Juan Councy, (Jcah 

: X: 38 Navajo and Anglo scudencs; grade 1; 53 
Navajo and Anglo scudencs 

: K: 46 Navajo and Anglo scudencs; grade 1: 55 
v Navajo and Anglo scudencs 

: 9 months 

: 5-7 years old (grades X-i) 

: Transicional bilingual educacion 



Descripcion 

tI^h°3h C Hh U i 1971) 6ValuaCed a Ungual program for Navajo scudencs in 
lit! T ^contained Anglo scudencs. Daca from an unspecified source 

Navajo scudencs. Navajo college scudencs were hired to forTbiUngual <n- 
scruction teams wich certified <non-Navajo) ceachers. Navaio lanSKe was 
used co ceach subject maccer and a Navajo hxscorv and cultural p»«2 
.nglish was caughc chrougn er English as a second language^ppro'ach! 

sch2l ZEST" f ™n 3 foraed , fro,n "udencs in che neighboring 

school d±«2~\ J 0 " 7611 aoCeS chac "udencs in che comparison * 

J2ool hours aL M ^i^P"^ children oucside 

SriL^ftn sss^ssr 1 3Cudencs had hiscorically — 

EngL^^l 3 S^% n "ntT C ::o P re r s:° n SCUden " *"« °° ^ ««! 

Screngchs 

The auchor used analysis of covariance co adjusc for preexiscinz dl«- 
sllTcltln " "»P"**» ^P 3 due P co"onrand S of 



Discussion 



n „rliJ 3cude °" from che comparison schools w er e hiscorically known co 
outperform scudencs from che projecc schools, che program effect^ 
probably underestimated by the analysis of cov.riance! Furthe£o« 
Coccrell did noc separace program effects on Anglos from chose on Indians 
These two groups were differentially distributed in che study: Indj - ans 
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Navajo 



Treatment 
Control 



76 
54 



15 
47 



and S£!„? b r e K Sh ° WS ^ Si « nlficaac difference in the proportions of Anglos 

not seS^ L:1nS CU?3 *-, 20W T r ' Coc =^'* ^ai y ,x ? s of co variance foe" 
not seen „o take into consideration this important fact. 
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Mame of Study 

Author and Dace 

Location 

Treacmenc Group 

Comparison/ Control Group 

Duration 

Ages 

Type of Program 



: The Effects of an English-Spanish Primary- 
Grade Reading Program on Second and Third 
Grade Students 

Huzar (1973) 
Perth Amboy, New Jersey 

84 randomly assigned Puerto Rican students 
76 randomly assigned Puerto Rican students 
2 to 3 years 
Grades 2 and 3 

Transitional bilingual education 



Description 

-Si!£ . Che ? r u gra !^ aS iaiciaCed in 1969, the subjects were randomly 
JSXl I < bilin ^ or regular classes, which remained intact 
'-rST 1 ^ ^« Primary grades" (Huzar, 1973, p. 34). The Incer-America 
3-?Lo ,1 ^""^u^ 1972 " 84 program 3Cuden " ^ ^ grades -2 and 

Lades 5 „T?r n h) ' C ° 76 C ° nCr0i " UdecC3 (cwo cia33 « 5 
grades 2 and 3). There were n initial differences between the two groups 

?he third ReadiCg Readine33 Te3C « on an IQ test administered' to 

Scanlev i P 0 """"™^ control group design (see Campbell and 
h ' a ^! y ; 1973) ™ u3ed > and che difference between the groups was tested 
oifUSE 2? P° 3 " esc ^ nl y control group design is perhaps che 
only setting for which the t-test is optimal" (Huzar, 1973, p. 40). 

f^L'S'J* 0 bi ^ n « ual classes' scores were averaged there were no dif- 
llliZlJ V h * Sr ° UpS ° n Ch6ir Ensli3h readiE « ski1 ^ w ^hiP the 
for SEE.? °° P f °°* Che Cla3S ha,,lng "Ungual teachers (one 

ror Spanish one for EngUsh) performed significance better in English 

teacher, but in the other grade there was no difference. 

rontlVT 7 analy3is of ^riance comparing scores by sex for treatment and 
control classes was significant. Although the control boys had the lowest 

ocTcT* 0 " 01 ^ f0Ur gr ° UpS < c " ac ™^ * sex), the Shefree test for 
pest hoc contrasts was not significant. Thinking the Sheffee test was -oo 

SSST^' ^ aU ^ h ° t Perf ° raed 3 C "" 3C ° n °» Sr-P 3 of bo^ and 
.ound the treatment boys scored significantly higher chan the control boys. 
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Nuuber of Soys 
Grade Treacaenc Control 

2 18 25 

3 23 18 



Strengths 

The methodology made use of longitudinal treatment and random assignment, 
with tests tor initial equivalence between the groups. Teacher effects 
were examined to a degree. 

Discussion 



The implication that transitional bilingual education was differentially 
effective for boys and girls is probably not correct. 3oth grades wa-e 
combined in the analysis and an inspection of the sample frame shows the 
following: 

There were more older boys in the treatment group than in the control group. 
Since older students would be expected to score higher, this distribution 
across grades puts a disproportionately higher number of higher scoring 
third-graders in the treatment group. Therefore, the result shown in the 
t-test comparison could indicate nothing more than the unequal distribution 
or boys across grades within the two groups. 

The limitations on generalizing from the study are that it covered a 
United number of grades (two) in one school district, and a limited number 
or subject content areas (one) in only one language group. 
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Mane of Proiect 



: The Follow Througn Planned Variation Ex- 
periment, Volume 4-a 

Auchor and Dace . stebbins ec al, ( 1977) 

T - ocacion : 5 sites 

Treatment Group s 492 follow through participants 

(64Z aoc white or blacic ethnicity) 

Comoarison/Concrol groug , 5 6* regu lar English curriculum scudencs 

*• (ISZ noc whice or black echnicicy) 

Duracion . 4 years 

MSi Grades K-3 

Type of Prograa . transitional bilingual education 

Descripcion 

The Follow Through (FT) experimenc covered 13 dlfferene lflst^M™,,! 

into che early elementary years. One of che 13 models -SEDL-vas a laT- 
ThlJilTJ? b c U T f 1 iascru « ion - Approximacely chree-quarcers of 
S£et22£V e " SP^^P'-Wng. Dual language skills were scressed in 
mosc curriculum areas. In che bilingual classroom model, che English- 
speaking children learned Spanish and che Spanish-speakers leaSd English 
The rollow Through evaluacion covered 3 cohorcs over 4 vears (K-3) lot- * 

"naUy s e ecci a ed 7313 ? ^ ? ° f ^ U ^ ^Iclf ~!echod~° 
--nally seeded on far che scudy and supersedes earlier, reports . 

The analysis is very complex. Fif ceen ouccome measured were used 

t£ To 71 " SCed *? ' diffe " nC "«i«ical models ac each s"e 
The 10 models vere various ^chods of adjuscing for che ef feces of non- 
random assign**™. Background covariaces included WHAT precesc score, 
iia",cctr?«?« iBC "». """^ipa, echnic group, sex, age, becveen-site 
characteristics, C ype of preschool, mother's educacion, and HAT score. 

r<«r JJf b *! i f anal ? 3is 3«^cegy was co seek consiscency among che sta- 
tistical models. Although each model incroduced unique artifacts into c^e 
result, consiscenc findings across models are probably roSusc andean be 
taken as evidence of crue effects. The anchors concluded^ 

The performance of FT children in chis model on che various measures 

comoari™ 00 ? C=6r7 Vari " con3idarabl y Cross cohort 

comparisons also vary among sices. In general, FT children far chis 
model perform as well on che cognitive conceptual skills cescs as they 
do on che effective measures; overall, 12 percent of these measures 

5lH "i^T"! 0 "; * child « n P«"*°™ somewhat differently in che 
basic skills domain, where 20 percenc of the effects are positive and 

wIchIn C oa «i^, nUU ; P03iSiVe " nul1 effec " cend co be concentrated 
within particular sices. This implies chac.che SEDL program has a 
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f !f"v tS . b7 Si "' rangia 8 100 percent positive 

?^L U Sm ? 6 153310 3killS dooain) Co 50 P« cenc «gaTi n 
Philadelphia, also in the basic skills domain;. Bock and 
Stebbins (1977) 

r„n ^ S ? L pr °8 rao 13 one °* but three American orojeccs we 'ound chat 

SSSS (3^79) ""The" 6 " aChinS J 1 ""^ ^•Ll 3 before e teSng h L2 
xiceracjr (SEDL, 1979). The program consists of a coordinated K-3 curr-cu- 
lua in keeping with the overall approach of Follow Through 



Strengths 



sites c^c^T V*^ Urge 3anple ^"ributed across several 

sures' fl™ "P licaci ^. and an extensive battery of outcome mea- 
sures, a very comrle^ statistical analysis was used, making this one of 
the more powerful of the nonexperimental studies. 

Discussion 

ized in he cab"f" 2 ! f *" ** (S " bbinS " i97?) ^ SiS a " s ~" 

tABLZ 2-4. SUMMARY OF FOLLOW THROUGH RESULTS - BILINGUAL PROGRAMS 



Sice 


Posicive 


Philadelphia 


0.67 


Los Angeles 


0 


Tulare 


51 


Sc. Marcin 


0 


San Diego (Texas) 


7 


Tocal 


• 11.9 


All Follow Through 


12.8 


Source: Scebbins 


(1977), p. 



Resulcs (%) 



Neucral 



Negacive 



88.3 


' 11 


94 


6 


49 


0 


89 


11 


79 


14 


79.6 


8.5 


67.6 


19.6 



findiSs SUm^ST i Che pr ° P °" ion « P°sicive, nejative, and neutral 
findings .rom 150 analyses per site (15 outcome measures by JO statistical 
analytic models). The 13 Follow-Through models in general had "SlS 
act in comparison to the regular school program, fhe sSl£oS2 was 

or ™. ^ ab ° UC f 3 effeccive * s "gular schooling with 80 percent 
of the tests showing no significant difference. Within the 5 SEDL sites 
one site Uuiare) stands out for its positive effects. t woulf not be ' 

e^ c °ive able Ho CO wev C rr Cl i U f if V " Chl3 one ^ite^at ^s 

ei receive. However, it is by no means clear that the effect can be at— *«.- 

fou\ other t?I Ual j: ,iUCaC i 0n 3in " Che ™ was ^ «puj e *V 

tour other sices where the proportion of negative results is far greater 

than the proportion of posicive resulcs. We could find no indication in 

Steobins et al . ( 1977), SEDL ( 1979), and Bock and Scebbins (W77) of wny 
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Tulare was so different from the ocher sicas, both within che SEDL node! 
and across all Follow Through aodels. 

la addition co che unknown sice-specific confounding, ;here are of-e- 
problea, or confounded creatines in che iaca. As a FoUow t£ou£ 
gram, SEDL also provided— 

o Medical and dental services, 

o Nutritional programs, 

o Social services, 

o Cuidance and psychological services, 

o Individual and small group instruction, and 

o Coordinated £-3 curriculum. 

^J ha 3C !iV f ChC S f? L P"^ 30 b « appreciated b 7 che program cos r.s 
^nich were S800 per pupil or 62 percent over and above che average pupil 

tultr %V*Z r , ea50 f abl y «P«- Co improved performance chroigh anv 
number or instructional methods if one was given a budget increase of 62 * 
percent. Given che budget available to SEDL (and to che other Follow 
Though Programs), che quescion can be raised chac since they had so much 
to wor2 with, vny did so little result? 

» 

Our iacarprecacion of che daca is chac' no reasonably convincing evi- 
dence ror che errecrivenesa of bilingual educacion is co be found in che 
caca • 
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Name of Study 



A Socio licguisitic Approach to 3ilingual 
Education 



Author and Date 

Location 

Treatment Group 

Comparison Group 

Duration 

Ages 

Type of Program 



A. Cohen (1975) 

Redwood City, California 

45 Mexican-American children in 3 cohorts 

45 Mexican-American children in 3 cohorts 

2 year study of a 3-year program 

Grades K-3 

Transitional bilin^ui!. education 



Description 

The theory often put forth justifying TBE stresses a sequence of initial 
literacy in LI followed by the development of literacy in the second lan- 
gauge. The Redwood City project is one of only three American studies 
where it is clear the prescribed sequence "was followed. Two program stu- 
dents and 13 comparison students were retained during the years of the 
study. Males were twice as frequent in the program as in the comparison 
group. 81 percent of urogram participants had parents who were bom in 
Mexico and 77 percent of the comparison group parents were bom in Mexico. 
Parental pe mission was required of participants. An extensive battery of 
tests were administered to measure language proficiency, language use, math, 
academic aptitude, language attitudes, socioeconomic level, educational 
environment of the home, and demographic -factors. 

Almost half the comparison students were receiving special assistance 
through Title I (24 percent), ESL (18 percent), or tutoring (4 percent). 

Across the three cohorts and the multiple-test battery, 100 F-tests on 
English proficiency were conducted. Of these 100 tests, 14 were signifi- 
cant with 11 showing superior performance by the comparison group and 3 
favoring the bilingual program 

Sased on ratings by parent^ and observers and on student reports, Cohen 
reports "The bilingual project did promote greater use of Spanish..." 
(Cohen, 1975, p, 226). 

One of the three cohorts showed superiority for the program participants 
in math and in gains in nonverbal IQ. 

Cohen summarizes the findings as "Mexican-American children who were 
taught the academic curriculum in Spanish and English for several years 
appeared to be as proficient in most English language skills as comparable 
Mexican-American children taught conly in English' 4 (Cohe*:, 1975, p. 163). 
"Th*. Mexican-American children following the bilingual program performed a* 
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well as, or better than, comparison children on cescs in a nonlanguage 
suoject matter, namely mathematics - (Cohen, 1975, p. 236). -The 3ilingual 
Project also had ao apparent detrimental effect upon the academic aptitude 
of the Hexican-Aaerican children involved. In fact , that program seemed to 
have enhanced academic aptitude in she case of the youngest grouo" (Cohen, 
1975, p. 237). ' 

Strengths 

The program is replicated through three cohorts. Analysis of covariance 
was used to adjust for the effects of nonrandom selection, following testa 
ror parallelism of the regressions and high within group correlations be- 
tween pre- and post scores. Longitudinal data and a large battery of tests 
covering IQ, English, Spanish, and oath developments were administered. 

Discussion 



The author interprets the results by emphasizing che point that lan- 
guage minority children can develop their home language in school without 
worrying about ill effects on English performance. However, Cohen began 
with the argument that initial literacy in U would lead to better levels 
of L2 skills. This hypothesis, which underlies the rationale for bilingual 
education, was not supported by the data- Indeed, if anvthing the data 
show some negative effect on English development. 

From our perspective, we find the programs' effect on English develop- 
ment to have been neutral to a little negative with mixed results in 
arithmetic. 
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Nan; of Study 

Author and Dace 
Location 

Study Populations 

Duration 

Ages 

Type of Program 



Final Report of the Compton Unified 
School Districts Title VII 3ilingual- 
3icultural Project, SeDtember 1969 
Through June 1975 

C. Stem (1975) 

Compton, California 

213 students in a Title 711 program 

1 year 

Grades 4 through 6 

Transitional bilingual education 



Description 

rh.^.iV 116 fina V e ? orc on » 6-year Title 711 project. The results of 
the first 5-year evaluations are summarized in insufficient detail to per- 

ii^s:ss: n d«au. fimling3 of che sixch evaiuacion ^ »■ 

*r !iV iC ^ e I 11 P5°^ ecc 0P«ated in one school in wo of three classrooms 
at grades 4, 5, and 6. A variety of information is presented and the re- 
port is noteworthy for the detail of program process information provided. 
Among the data presented are norm-referenced analyses, analyses of sains 
.or participants with no controls provided, and grade-equivalent scores. 
As discussed in chapter 1, 'we can make little use of this data. However, 
the author also presents an analysis of covariance of the California Test 
ILrZL iwJ and i M subces " for 8X' a d« 4, 5, and 6. When averaged 
sell" S rh^f ChC Pr0gra ° effecC waa native on all 9 component 

JZZ *< 5« * CT8 f\ ^ 0iang " r " ul " across *™ dM . there were 4 nega- 
S a ^ **' 5 ChaC W *" n ° diffe « nc S^ade 4; 5 negative results 

£ ^f^c^";! ^ gra<lt 55 ^ 8 ° e8aCiVe " 3U1 " «* 1 Ch " 

Additional data are presented showing that students vho have been in the 
program 3 or more years score higher than students who have been in 2 or 
fewer years. r 

Strengths 

Stern employs some longitudinal analysis and adjustment for the effects 
of nonrandom selection by analysis of covariance. The program she studied 
had been operating long enough to become stabilized and involved a larz- 
number of participants. s ~ 
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Discussion 



Generalization is limited by the fact that the study took place in only 
3\2E? iD Ch "\ S"' 1 "' To a .^gree, the results are incousL tent 
if longer exposure to the program leads to better performance, why did the 
controls do better? There are two interpretations for this situation. 
First, the analysis or covariance adjustment may not have been able to 
overcome the severe selection bias chat occurs when cvo-ehirds of the stu- 
dents are placed into the program. Therefore, the longitudinal analysis 
is correct. Alternatively, so much time is taken away from practicing 
English when students first enter the program that scores are denressed. 
As erne passes and ooze English is added to the curriculum (as is charac- 
teristic of the T3E program), progress is noted. Nothing in the data en- 
ables us to select be ween these two alternatives. 

f a ™J laa S\ Se " a n ° CM ChaC a£ter 6 y, * r> of P"* 130 operations, some 
crStL a r bS Paat ? w ? r V nClUded iQ Cha coo P" ison group. This situation 
-SS^ted- £ Q f ?i Pr0graa " Che f ° raer P*^ciP«" had been 

graduated from the program on the basis of improved performance. Un- 
fortunately, Stern does not examine the possibility of such an effect's 
haviig occurred. SI * ec " 3 
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Same of Scudv . 2S2A T1=JLa ni 3ili3gual ? rogram 

Author and Dace : Cars rod and Curtis (1980) 

Location : Austin, Texas 

Treacmenc Group , Grade 4: 80 Ticle ni ?rogran scudenC3 . 

Grade 5: 92 Title VII program students 

Comparison/ Control Group : All other Mexican-American students in 

school district 

Duration : 5 Tears 

M££ : Grades 4 and 5 

Type of Program : Transitional bilingual education 

Description 

Carsrud and Curtis (1980) studied the change in percentile scores from 
whe cirst grade for students in a Spanish-English bilingual program. " Thev 
compared students who had been in the Austin, Texas, Title m project for 

dJL Jnlll ^'''J^r" aeVer in Che pr °J ecC and *l Hispanic stu- 

dents in the district (see table 2-3). A significant proportion of the 

fiffh 1 ^ ° V " ° ne " half f ChB Pr ° j ^ C f0U " h *" de " and'one-tJird of the 
•!ere \S de h!i T" aoaol3 : a ^ zl ia EhgUsh. Many of che other students 

h „!! S T ! «udencs. Thus, the majority of students served by 

the program were either English dominant or English-monolingual. 

Comparison of gains for project and nonproject students were carried out 
by regression analysis with project .status and pretest score as predictor 
variables. A significant difference! in favor of the p,ojecc was 'found In 

£ J*lr 1 ^ b " n in ^ pr ° jeC= for 5 ««i«elv« years, but 

no dlftarences were found for English or for both English and math in 
scudencs who had been in the project continuously from grade I. 

Strengths 

i,„2fnr d CurCi ^ out a lon gi~dinal analysis wich pretest ad- 

justment to correct for effects of nonrandom assignment. 

Discussion 

Although the longitudinal nature of tho data is a plus, there is no 
doubt chac chere was attrition over the 4 to 5 years covered by the scudv. 
It would have been very desirable if che Puchors had provided an analysis 
of che effeccs of accrition. ' 

It is worch reproducing Carsrud and Curcis ' data cable (see cable 2-3) 
Co illustrate the magnitude of the oroblem encountered in teaching many 
language-minority students. Mosc Mexican-American scudencs began school 
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at the national, nonn level* Note chat in 4 or 5 years the Mexican- 
Aaarican students' performance has declined about 15 percentile points 
whether or not they were in the bilingual program* 



TABLE 2-5. PERCENTILE SCORES OF EXPERIMENTAL AND CONTROL GROUPS 



1980 California Achievement Test 
Grade Scale 



First Grade 1980 

Percentile Percentile Change 



4 
4 
4 



5 
5 
5 



Reading 
Reading 
Reading 



Math 
Math 
Math 



Reading 
Reading 
Reading 



Math 
Math 
Math 



Project 

Nonproject 

District 

( Mepcic an-Ame ric an ) 

Project 

Nonproject 

District 

(Mexic an-Ame ric an ) 

Project 

Nonproject 

District 

(Mexican-Ame ric an ) 

Project 

Nbnproject 

District 

( Mexic an-Ame ric an ) 
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46' 


-8 


54 


39 


-15 


51 


39 


-12 


50 


31 


-29 


52 


38 


-14 


63 


39 


-24 


59 


32 


-27 


57 


43 


-14 


67 


38 


-29 


55 


33 


-22 


58 


39 


-19 
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— o£ Stud 7 Evaluation of the Impact of ESEA Tide VII 

Spanish/English Bilingual Education Program 

Authors and Dace : Danoff ec al. (1977, 1978) 

Location : National sample 

Treatmen t Grouo ? s 300 ti-i* vtt c « „ v u , . , 

. j f QKjKj ii u j.e viz Spanish-z.ag lis h proiecc 

students 



Comparison/ Control Group 

Duration 

Ages 

TTP* of Program 
Description 



2,400 non-Title VII students 
6 months 
Grades 2-6 

Transitional bilingual education 



The American Institutes for Research (AIR) carried out a national evaiu- 

(££f f°\t h al i ^977° f 197lt "S' I? ********* biling^ ro^rS 
uanorr et al. 1977, 1978). The first results were released in February 

1977 and caused a stir in the educational community because the research 
l^ol%^ effeC " SCUden " P*™^ in 

n 0 n-?? C ^^ P J e /° n3i " 8d ° f 5,8 °° pr0jecC in 37 sices and 2,400 

STSlI^ rl * C ° aCroUed for "hnlcicy, socioeconomic status, and 
grade level. The programs were in their fourth or fifth year of operation. 

Results of ch* study indicated that Title VII students did slightly 

»reaa" triSLcf 2"!! 3Ur? r s « d Che con « 01 *™>P- In response to vide! 
spread criticism of the study from the bilingual community and the National 
Institute or Education, additional data were collected onVsubsampS of 
the original universe. The subsaaple was posctested after a lon^r creat- 
ine interval than che original sample. The additional data were exten- 
sively reanalyzed by AIR (Danoff ec al., 1978). This addicioual anSvsis 
somewhat changed che original conclusions: che rean«l y sirloSn7n^£|«- 
ence in mach scores between che program and comparison scuaencs and con- 
tinued to find superior English performance for che comparison gSup? 

* Ch ,?\ clonal no ™». boch group, were in che boctom rifcTof che 
nacion in English and che boctom chird in mach. 

Strengths 

,„7^%f UCh0rS dr6W V' 17 large saaple and had a «H-conc rolled daca 
collection program. The scudy included excensive sophiscicaced sca-iscical 
analyses including analysis of covariance to .djust for preexiscS defer- 
ences due to nonrandom assignment and a scacistical tesc to determine 
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^"t er Chere was « underadjuscm* problem with che analysis of 
covarxancc. 

Discussion 

methodology used in conducting che study has been critically ILllttt w 

of ^52 crlrS? 83 ? *" " Ud7 C0 ^ lecei y invalid." The key elements" 
of .hese criticisms and our responses to thea are presented below: 

l * ig! P fy; SOn f r ° SS **SES2.* Bilingual education is very complex 
S^ 01 ? raU * d c ° acknowledge that bilingual programed." 
velop and exist under varied conditions due to specific district 
characteristics: linguistic needs, availability of quaJUfiei 
teachers, adequacy of curriculums, district commitmentTand po- 
litical underpinnings (Gray, 1977). The AIR study failed to 
recognize these differences and treats bilingual education as an 
SSf £ ! renCiaCed Md uaif °» P"8T3«. As a result, TsilU 

ffftctfofa T 0 THu CUC 1 b7 3117 n8gaCiVe flndin ? s 50 
errects of a good bilingual program are lost. 

ar . tt ? f icicism that it is unfair to compare across programs does not 
S:i e g ^ th8 , MCdS ° f P 0 ^^"^ «» make informed L5sions based on 
ct " nS " s V : £f t C c a : ff»^<i-al bilingual education is generally 3- 
hnrh *, i ? ^ Sh0w up on che average. The AIR study refle«s 

both good and bad program*, ^ o£ suffer froo ««1 tSle^nL 

aitLL r °«ne 3 cC and reS r!r CansC « inCs - however, these Sntrain s'and con 
2r££I* aCCUal dlsCricc and school problems which determine how 

effective a program can be. Such programs muse be included to measure the 
effectiveness or 'bilingual education. measure the 

2- Testing Interval. Only 6 of the 37 projects involved in che 

study were cested over more than a 6-month period. The remainder 
of the projects were analysed for program results over 
periods. Evidence of cumulative gains in bilingual education' 
over several years indicates that such short periods wouJ Toot 
allow observation of the real long-cerm improvements due to 
transitional bilingual education. 

orig^'un^e: 19785 ^ * « 'he 

The results can, it turns our. h» « rlzed 3uc elneclv , 

For both the grade 2 cohort and ch e grade 3 cohort, r hl 
fall-to-fall achievement gains in' English Readin g <~ 
ggcnemacics Computati on in Title VII projects were-^ Th^ 
significantly nor substantially different from w har unniH" 
have been expecred vtcnout Tide 711 treatment, m rh nn . ' 
possible excepcion: che grade 3 cohort of Title'TlT 
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children in communities t or which no comparison classro oms 
were available (i.e.. in che urban Northeast r egion) shawH 

substantial gains in Ma thematics Computation skills . In 

light or these trends across methods , the signiii^nl 

dirferences favoring non-Title VTI in some of the grade 2 
analyses of covariance were more than likely due to that 
method's tendency to undercorrecc preexisting group 
differences in 3ome situations. 

Although the subsample is not totally representative of the original sample 
excluding Southern California), the AIR study still is the most comprSen! 
sive study undertaken of bilingual education. comprenen 

over T^nth UltT *£ 1 i CeraCure chac * *>otad at reported gains 
lTL .JlX? ^ °T 3 6 !DOnCh peri0d gains shouid *± s ° be made. 

r ^ 7 Ch3 u 3 3aDple 33 Urge 43 chac of che AIR study would have 
failed to detect short-run gains if they were made. 

3- Teacher Qualifications. The AIR Teacher Bilinguality Scale 
indicated that only half of the teachers involved in Title VII 
projects were proficient in English and Spanish. Only 26 per- 
cent of the teachers participating in the study had bilingual 
teaching credentials. 

rm<«nA S 3 Pa ^ iC7 -° f ^l"** bilingual education personnel in the 
United States. Thererore, that AIR found only 26 percent of Title VII 

SstnT ^s^fLr 1316 S be qUal±fied ^l^-l ? educators Sttlur- 
SiiSr, 1981) r «leocs the actual situation in the United States 

on iIIaZV' ^ anal £ is assured the impact of teacher characteristics 
«L P« f ormance. The study indicated chat formal credentials were 
not related to performance. 

4 ' ^f 0p " T C0n T ar ^ 30n Group - asked ^e principals of che 

Title VII schools co idencify nearby schools wich similar scu- 
deac bodies wichouc Tide VII .programs co form che comparison 
sample. A number of schools were unable to identify comparison 
schools. Furthermore, there were initial differences in -he 
level of language skills between the Title VII group and the 
comparison group. 

This last criticism does net recognize AIR's use of analysis of covari- 
E?i 5 ad l Me 5" preexlscin « differences between the rwT^oups U we 
conclude chat, despite their efforts, AIR failed co produce an accepcabS 

?^ir° n chen we «»" a l so ^1 che scudies chac employed 

inferior mechods for identifying com. arison groups. . ? ' 

Rossi (1979) and NIE (1979) have analyzed che encire AIR projecc. While 
recognizing chac che AIR scudy is noc wichouc problems, chey must be Jo"c 

valTcSS'wnln R ° 3 f acka ° Wled « ed chaC « «=he problems' 3Lr2t^2th 

evaluacions when assignmenc is nonrandom, AIR did an adequace job. Thac 
Che scudy may noc have evaluaced poorly inplemenced projects does not 
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decracc from tcs conclusions as co Che effectiveness of Tide VII programs 
It may or may noc limit drawing conclusions abouc cransicional bilingual 
education. The AIR study is one of che besc existing nonexperimental scud 
«* 3 uf ^ U f n « , i al «ducacion and ics conclusions muse be given some wtighc. 
we think che r-Howing conclusions are supported by che MR study: 

o Title VII programs have not been shown co iaorove students' per- 
roraance in school. Math scores seem unaffected but English" 
performance is worse. 

o The largest, most comprehensive study of a bilingual education 
program ever undertaken found no evidence that the program is 
an effective way to meet the needs of language-minority children. 
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Maoe of Study 

Authors and Dace 
Location 
Treatment Group 

Comparison/Control Groun 



Duration 

Type of Program 



; 3ilingual Education of Children: The St, 
Lambert Experiment 

: Lambert and Tucker (1972) 

St, Lambert, Quebec, Canada 

: 64 monolingual English-speaking children from 
St* Lambert, Quebec; cohorz I: 26, cohort 
II: 33 

: English Control I Group: 43 monolingual 
English-speaking children ia the same 
school, in a regular English curriculum; 
cohort I: 22, cohort II: 26 

English Control II Group: 54 monolingual 
English-speaking childroc from Montreal, 
in a regular English curriculum; cohort I: 
26, cohort II: 28 

French Control Group: 47 children from French- 
Catholic school in Sc. Lambert, in a French 
curriculum; cohort I: 22, cohort II: 25 

: 4 years (longitudinal study) 

: Grades 1-4 

: Immersion 



Description 

The prototype immersion project is the St. Lanbert program in French 
for anglish-speaking students in Canada (Lambert and Tucker, 1972). l n 
the St. Lambert project, middle-class, monolingual speakers of -nglish 

5S„™f?° dUC !? Ft "? Ch iD , " lamr,ion kindergarten program caught by 
bilingual, nativeFrench-speaking teachers; students continued to receive 
monolingual rrench instruction through the first grade. In the second 
English language arts course was introduced for 1 hour a day. 
Ultimately, 40 percent of class instruction was in English, including 
English language arts, art, physical education, and music taught bv a 
native English speaker, sixty percent of class instruction was in' French 
and taught by a native French teacher. Although instruction was delivered 

4 rT Jr*?? 1 :! scudencs could 3 P es * Co *nn ask questions of the teacher 
in Li. Unglish). The teacher, however, always answered in L2. 

.^. 7he " K?f 8 u C T CyPe ' ° f con P arl3o n groups. Two regular English classes 
'ere established as controls. They received instruction in English, except 
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.or 1 hour or instruction la French as a second language (FSL) each cay 
A _ trench curriculum class also was chosen as a control. This class received 
cne regular French curriculum for French schools in Canada. The groups were 
careruliy compared for equivalence using socioeconomic status, iq, language 
achievement, and home background factors based on interviews la the home. 
(The children involved in. the study came from riddle- and upper-aiddle- 
class homes.) rr 

The St. Umbert atudy followed tvo cohorts from kindergarten through 
grade 6. The results showed a general trend of 12 performance superior to 
tnat or the English control group and approaching that of the 12 
aonoliaguals . 

ii J*? !!** Fr " nch ' buC C6SCS Were ^ van in bcch ? " n <* and Eng- 

lish with no difference in scores resulting from the language of the test. 

English (LI) performance was initially depressed during the first 2 v-ars 
when there was no formal English instruction. However, when English in- 

no'SSve" £veis? 0dUCed ** ^ 3Ch ° o1 U SCoreS ^ zoved c ° 

At grade 4, che experimental immersion group wa° equal to che English 
controls in subject area achievement and intelligence, but slightly lower 
in English oral and listening skills. Their achievement scores in French 
were average compared with Montreal norms; their scares in oral skills 
were slightly lower than those cf the French control group. The immersion 
group was, retested in grade 6. The English control group and the experi- 
mental group had equivalent scores on the English exams administered; but 
-he experimental group was not equal to the French controls on the French 
k ^f 3100 students, although having made significant strides, 

™^ ?T" °* flu,nC7 > n Ch€ l*"Su*««. However, the experimental 
groups L2 performance was far superior to chat of the English controls 
receiving French as a second language instruction. 

Strengths 

Following the progress of two cohorts permitted replication of che re- 
sults. Considerable background information was gathered on the students 
so that comparability of the groups could be tested, and covariates were 
used in the analysis to adjust for initial differences. An extensive 
bac.ery of tests and comparison groups were used, providing considerable 
information on the students' development. The longitudinal design aade ic 
possible to determine long-term trends in development. Finally, the over- 
all pattern of results is consistent with the authors' theoretical propo- 
sitions and is difficult to explain by any more parsimonious alternative. 



French as a second language is the equivalent of English as a second 
language when LI i.i English rather chan another language. We therefore 
.nclude rSL as an ESL program, since that makes the terminology consistent 
for the American case. 
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Discussion 



We have some concerns about the methods used in the St. Lambert study, 
first, the sample size was very small, especially for a longitudinal study 
in which normal attrition (students moving in and out of a particular 
school) can be expected to reduce the sample size over the years. The 
first experimental cohort contained 26 students, reduced to 20 by the 
fourth grade; che second cohort began with 38 students, but dropped to '7 
by grade 3. The voluntary nature of the program raises snecial concerns 
about attrition, since a plausible alternative hypothesis' would be that 
students who were doing poorly in the program were oulled out by their 
parents, thereby biasing the program with students who, for whatever rea- 
son, did rather well in learning French. In addition to attrition in the 
treatment group, attrition in the French-speaking control group was so 
high by the second year of the study that the authors felt compelled to 
supplement the comparison with the less deeifable norm-referenced design 

Program participants were all volunteered by their parents. The first 
chapter or this report discusses how such a process can introduce bias* 
into a study. This type of bias dould have been especially acute in the 
St. Lambert study, since the program was begun by the school only after 
intense parental pressure for -a special program to teach Canada's official 
second language to their children/ Recognizing the possibility of selec- 

"-^J^"' i"i a r TuGkar * a " nered considerable preprogram data on the 
students and their home environment and found there were differences among 
tne various groups on several parental attitudes and on two of five indi- 
cators of socioeconomic status/ The authors concluded these were primarily 
difrerences becween the French- and English-speaking comparison groups, 
since the treatment group mean fell between the means of the other two 
groups in the analysis, of covariance. However, to have been completely 
certain or this interpretation, the authors should Lave carried out pose 
hoc contrasts of various pairs of means. It is important to note there 
were no differences in nonverbal IQ among the various groups of students. 

English performance was depressed at the end of the first grade relative 
to the comparison group, but equal at the end of the second grade. The au- 
thors attributed the improvement to a facilitating effect on learninz to 
read a second language (English, in this case) from having first learned 
to read another. This argument overlooks the fact that the students came 
from English-speaking homes where parents were concerned that both languages 
be learned. These children had 40 percent of their school day (including 
classes in art, music, and physical education) in English. Sixty minutes 
of formal anglish instruction per day in the second grade could have been 
enough to bring children of this background up to par without any facili- * 
tating effect. 

The authors also tried to demonstrate language transferability, follow- 
ing a study of bilingual college students who were better able co discrim- 
inate sounds in a third language than monolinguals were, the authors in- 
cluded a test of discrimination of Russian phonemes at che end of each 
grade. No difference was found becween the program and comparison students. 
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Despite this direct rejection of the transferability hypothesis, the authors 
proposed -a trans rerability explanation for several results. The authors 
also played down the depressed English performance of students after grade ' 
and the occasional negative effects indicating mental confusion, which ap- * 
peered in the yearly IQ tests. * 

In reviewing the IQ test results on the Lorge-Thorndike "not-belonginz" 
subtest, Lambert and Tucker (1972) pointed out the mean for the experimental 
class was above the mean of the controls. Although, the authors argued than 
this measure shows the experimental children reaped benefits, the difference 
was not statistically significant and was only about one-half the magnitude 

t it 3CaC f" icall 7 significant difference favoring the compari sodden ts 
on the vocabulary subtests, a difference the authors dismissed as worthy of 

ao special attention" (Lambert and Tucker, 1972, p. 123). 

Q f ^.^fr 8 c ° ncluded ch * re waa "m "«=ive language retardation deficit 

iL^ll " d TuCk * r ' 1972 » p - 152) ' cwo "sts of English 

proficiency showed the comparison students to be significantly superior to 
those in the program (see Lambert and Tucker, 1972, pp. 147-48). 

If /^J^!","* J UCk f r ! tUd7 aa0Cher im P° rtanc ^Plication for the 
A'MrM^^ ^ 11 ! *? b8 ! C """P^^ed in language-segregated settings. 
A -ittle-noted substudy in the Lambert and Tucker study looked at the con- 
sequence or putting some native French speakers into the French immersion 
classrooms. It was expected these students would provide French-speaking 
role -odels for the students learning French and would improve cheir per- 
formance. However, just the opposite happened. French (LI) Performance 
of the immersion students declined. Apparently these students could not 
keep up with the native French speakers, who > ended to monopolize the 

IT till V 1 ? 6 ' ?' im P licacion is 'i*"' Language-minority students, 
at least during the early stages of acquisition of English, should be 
separated from their English-speaking peers. 

f? 3 E iC ! 3008 cechn i cal Problems, the study is impressive. The degree 
to which the St. Lambert experience is generally, applicable to the United 
states, however, is unknown. The two settings differ in the following ways 
(see Paulston, 1978; Tucker, 1980) and perhaps others: ^xowxag ways 

o St. Lambert was a middle-class, suburban community. American 
bilinguals are almost exclusively of lower socioeconomic status 
in either rural or urban communities. 

o L2 was a high-prestige language in the community. Most American 
communities place a high value only on English.' 

o LI was the majority language of the culture. In the American 
setting, LI is a minority language. 

o The goal of the project was to develop bilingual students, persons 
fluent in both English and French. The emphasis of American 
policy is on English mastery. 
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EUne of Scud^ : three Year Evaluation of a Urge Scale Early 

Grade French Immersion Program; The Ottawa Study 

Authors and Date : z^rik and Swain (1975) 

Treatment Group and 
Comparison/ Control Group 



Grades 






Cohorts 
II 


III 




Exp, 


Cont. 


Exp. Cont. 


Exp, Cone „ 


X 


220 


200 


270 310 


140 130 


1 


200 


225 


160 170 




2 


108 


120 







Durac - on : 3 years (longitudinal study) 

iS£i : Grades rt-2 

Type of Program : immersion 

Description 

A second major evaluation of a French (L2) immersion program was carried 
out in Ottawa by BariJc and Swain (1975), In this program only L2 was used 
.or instruction in kindergarten and grade 1. in the second grade 1 hour a 
day of English (UL) instruction was added to the curriculum. The study 
spanned 3 years and three cohorts in several schools « 

The study design was much like that used by Lambert and Tucker (1972). 
Students were tested in both languages and comparison groups received 15* 
to 40 minutes a day of L2 instruction in an ESL-type setting. The studv 
found that kindergarten immersion students did better in L2 after 1 year 
or instruction than did ESL-type students after 2 years. LI performance 
was depressed during the first 2 years, but recovered in the third year to 
normative levels. L2 performance of the immersion students in all grades 
and all cohorts was far superior to the comparison group and approached 
the national norm. (Since the students were around the 70th percentile in 
LI, median performance in L2 indicated they were still somewhat deficient 
in L2 given their ability level.) Unlike Lambert and Tucker (1972), 3arik 
and Swain did not include a native-French-speaking comparison group but 
used test norms instead. 

Strengths 

Barik and Swain had a large study population about whom they gathered 
extensive data. Students were nonrandomly selected for the treatment 
groups and comparison groups, but preexisting differences were adjusted for 
by analysis of covariance. By following several cohorts of children, Bar 
and Swain were able to gather longitudinal data. Program results were 
replicated in all three cohorts examined and were consistent with the 
findings of Lambert and Tucker (1972), as well as immersion theory. 
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Discussion 



3arik and Swain (1975) do not explain how students were selected to 
participate in the program, which raises a question of whether selection 
bias was present. However, their use of analysis of covariance 
is the standard procedure for taking into account the nonrandom selec- 

A f eCOnd rea3 ° a Why selecti0Q bias might not have been a 
problem is that the pattern of results is inconsistent with any of the 
alternative explanations based on selection bias. Selection bias holds 
™n a ^ am 3tuden " ?" i^cially either superior or inferior to the 
£?£~ ? g ^° UP 3C ChiS difference continues through the evaluation. 

Neither or these patterns is found in Barik and Swain's (1975) or in Lam- 
bert and Tucker's (1972) immersion studies. 

Tu,Kr" B " ik / nd Spin's study design was similar to that of Lambert and 

J J? ? ° Ur * eneral comments about Lambert and Tucker also apply 
™S Juf ^"af «ception. By drawing on an extensive experimental 
program through two large school districts and by selecting the comparison 
TJXl ?r none *Pf inenC ai classes throughout the districts, Barik and 
Swain had a much larger sample available. Unfortunately, the authors 
present no analysis of attrition effects, which occur in longitudinal 
studies to some degree. 
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Name of Study 

Authors and Dace 
Location 
Treatment Group 
Comparison/Control Group 

Duration 
A£es 

Type of Program 



English-French Bilingual Education: The 
Elgin Study Through Grade Five 

3arik, Swain, and Nwanunobi (1977) 

Elgin County, Ontario, Canada 

73 monolingual English-speaking students 

79 monolingual English-speaking students in a 
regular English curriculum; other control 
groups, including full French immersion 
classes 

5 years (longitudinal data) 

Grades 2-5 

Immersion 



Description 

a c^t£i e ?Ji* J 1977) " POrt ° n Che fifCh 7ear of a longitudinal study of 
TuSS n9%?Zl °* £v 0gra ?'c ^ P^grams of Lambert and 

tucker (1972) and Barik and Swaia U975) began with 100 percent of the 

iflliS? in (L2) f ° r ***** K and X > ch * n gradually shi? ed to a 

60 to 40 percent division between L2 and U (English) at the upper ele- 

TtoZJEl? I'" ^^° nCenC 3UbjeCtS CaUgh ^ in * chis pa? illlLer- 
ffiS?^ a! ^° langU3ge 3pliL of French «* Ea * lish befcinST 
SteSon^ UngUage WaS U " d in Che mmi **> ch * other In eh? 



a bi1i C ^l C ^ C n;1 mU ! iC, u F " nCh lat *«*« «" were taught in French by 
a bilingual anglophone teacher. Science, beginning in' third grade was 

' a ?* hc in ^Vlinh. The curriculum content was the 

to S *L Sn 3 regUlar EngUsh Pr ° gram - Spinning in grade 3 

« Ltif uT*** c 2»»?* at was Cau * hc ^ a different teacher with native 
or nacive-like command of French. A e 

(n I h 73r rea w«? r !U P C ° COn3i3C ° f four cohorcs ° f ° ne dass each 

~rJ n r». ? ? pe8 ° f C0B P ari30n groups were used. To assess LI 

performance, each cohort was paired wlch one class of scudencs in che ree- 

SL^STS^S'S Pr ? gram fr ° n 3 3inilar 3Ch001 ln < h « ^ «hoo1 
in another school r ° PMnt T aS3e33ed * c »P"i3on wich scudencs 
Lr ?!^ u district (apparently che data reported by Barik and 

tZt «S J < ? b ! Ch 3 f,lil PW and regular LI Instruc- 

tion wich one period a day of formal instruction in L2 language arcs. 

y^fZll 2r!°I" r f MCa U3ed non r rbal I( 5 «* «8« as che covariaces. 
year s daca were analyzed separacely, alchough che nacure of che daca 



Each 
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suggest* chac a repeated measures design nay have been nore appropriate 
(see also Lambert and Tucker, 1972; Barik and Swain, 1975). 

Strengths 

In the Elgin study, 3arik et al. had a large sample size about whom ex- 
tensive data were gathered. Several different comparison groups were used 
to measure different achievement levels. Students were randomly selected 
for the comparison groups, and for preexisting differences were adjusted 
.or by analysis of covariance. By following several cohorts of children, 
3arik et al. were able to gather longitudinal data by replicating the 
treatment. 6 



Discussion 



The results are complex and are most easily understood by reproducing the 
authors summary tables. Apparently, the data on L2 performance (cable 2 in 
the report) were not adjusted by the covariates and ( statist-leal c«t* *r» 
not reported. Therefore, we cannot accept the results of the L2 data and 
will limit our discussion to math performance. 

The authors note that understanding the math results requires an under- 
standing of the English (LI) results. Apparently in response to initially 
poor performance in LI (relative to the students instructed monolingually 
in LI), the school moved to strengthen LI instruction by reducing the time 
given math instruction (math instruction was all in L2). Further, the 
students received less total exposure to L2 than did the total immersion 
students. Therefore, it may have been more difficult for them to maintain 
normal progress in L2 math. Nevertheless, while the comparison students 
performed significantly better in math in 8 of 88 comparisons, 80 of the 
88 comparisons show that the students who were taught math in L2 were not 
.ailing behind in math performance by comparison to those taught in LI. 
Since the treatment group did not fall behind in math skills on 90 percent 
of the comparisons made, it appears that instruction in L2 did not impede 
subject matter acquisition. 

Assessing the study's design is difficult. On the one hand, one oust 
admire the authors for the ingenious way they patched together a rather 
comprehensive design from various sources. On the other hand, exactly 
how this patchwork approach may affect the results due to preexisting 
dif rerences is unknown. Neither is it clear that using age and nonverbal 
iQ as covariates can fully control for any (unknown) preexisting differ- 
ences. The inconsistent results within grade across cohorts and within 
cohorts raise further questions. The authors may or may not be correct 
in their speculation that these patterns reflect changes in school policy. 
Since this is a major issue, it would have been better if the authors had 
presented some hard data showing chat these policy changes had indeed 
taken place rather than merely speculating on the point. 
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TABLE 2-6. TREATMENT AND COMPARISONS SCORES IN L2, 
ELGIN FRENCH IMMERSION PROJECT 



Perxonnance in French, Grades 2-5" 



French Comp. Test 
Level 1 (max. -45) 



Test de Rendament 
en Francais 
Gr. i Level (max.- 
30) 



Toronto 

Elgin PFI* TFI a ** Ottawa TFI/RE*>*** 

-^-2 SE^J Gr. 4 Gr. 5 Gr. 1 Gr. 2 Gr. 3 Gr. 4 

12.47 26.55 27.29 36.31 31.69 TFI; 38.39 40.03 

RE: 10.58 14.20 12.15 



Gr. 3 Level 
(max.- 30) 



10.94 



8.71 11.87 17.59 



19.10 



TFI- 



15.96 



Gr. 4 Level 
(max.- 40) 



13.87 



26.06 



Test de Lecture 
Gr. 2 Level 
(Max.- 19) 



Gr. 3 Level 
(max.- 28) 



Gr. 4 Level 
(max.- 42) 



10.50 



11.44 15.73 21.50 



16.79 22.38 



12.83 



19.73 



28.82 



a Data taken from Barik and Swain, 1976b (unit of analysis - individual), 

b Da ir*n2 e ? 1°* 8 ? rik / nd Swain » 1977 of analysis - class). Amount of 

French instruction in RE program - 20 to 40 minutes a day from kindergarten 
• on (varies among graces). 6 e 

* Partial French immersion 
** Total French immersion 

***Total French immersion/regular English curriculum 



SOURCE; Barik et al., 1977, p. 465, 
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Name of Study 
Authors and Dace 
Location 
Treatmenr. Group 

Comparison/Control Group 

Duration 
Ages 

Type of Program 



: ABC's: McAllen's Immersion System 
: Pena-Hughes and So Us (1980) 
: McAllen, Texas 

: 78 comparably limited Mexican-American 
children 

: 78 comparably limited Mexican-American 
children in transitional bilingual 
education classes 

9 months 
Kindergarten 

Immersion, transitional bilingual education 



Description 

An English immersion program for Mexican-American students began re- 
cently in McAllen, Texas, and has reported gains in test scores, self- 
concept^ and discipline (Pena-Hughes et al., 1980). The project is differ- 
ent from the previously mentioned immersion programs in that the students 
are of low socioeconomic status, there is little parental involvement, it 
is not a voluntary program, the school is in a rural area rather than in 
a large urban area, the children are not monolingual but are comparably 
limited in both languages, and they are language-minority children, not of 
the majority culture. 

The project was begun in kindergarten during the school vear 1979-30 
Teachers, aides, and students (n - 156) were randomly placed in four ex- 
perimental classes (n - 78) and. four TBE control classes (n - 78). Stu- 
dents were controlled for IQ and socioeconomic status. Students in the 
experimental classes were Caught the same curriculum as the control stu- 
dents but their teachers spoke only English to them from 8:30 to 1:30. 
Students, however, could speak in either language. Physical education and 
cafeteria personnel spoke only English to the experimental children. Span- 
ish language arts was taught from 1:30 to 2:30 in the afternoon. 

Test results on the Language Assessment Skills (LAS) test (a Stata- 
approved language proficiency test) indicate that students in the experi- 
mental classes have made significant gains over the controls. Even though 
students were randomly assigned, the authors carried out tests to determine 
pretest equivalence and found the experimental group scored significantly 
higher on the Spanish pretest, with no difference on the English pretest. 
Therefore, pretest differences were adjusted for using analysis of covar- 
iance in order to assess program impact. In English oroficiency the con- 
trol group made a gain of 30. 23 points from pre- to posttesting while the 
experimental group made a gain of 43 points. 
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•i-S*Sil? in SpaniSh k ?roeicienc 7 «* ch « experimental group were also 
significantly greater thaiy those of the control group. Children in the 

gained 24. Both the experimental and control grouos made simificanr 

£ SLS J f Ch ^ dren iQ Che ^« ion could be identified 

fL, !t , Pr T U3e ° r Engli3h 33 COm P ared with = he s ™dents ^ the bi- 
lingual education program. 



Strengths 

The study was a true experiment in which both teachers and students 
ZEV^ 0 ? 7 ^ C ° Che B "° CreaCM « conditions. Although only in 
large lon 8 iCudin ^ and che sample reasonably 

Discussion 

cio? e i„ TB ^ P r gram U3 ! d u he aFPr ° aCh orally mailed- concurrent transla- 
SS'li t? w u PP u°*S Che CeaCher ^diately follows statements initially 
made in LI with the L2 translations or -vice versa. We argue later chat 
this approach is counterproductive because students cune out the language 
chey lease understand and, in effect, receive only half a day of inac- 
tion. Therefore, any conclusion drawn from the McAllen study that immer- 
sion is superior to T3E must be made conditional: immersion was shown to 

-o£K ~V° T r Ch °K bilinSUal edUCaCi0n ChaC a venues- 

c^onab-a pedagogical technique. < ' H 

.h a „S!%if?" era ° f r ff^ 3 C ° Uld 4180 be due Co che ^«horne Effect, al- 
I? "T ™ likel y b" 8 "" both groups were in their first year of 
school. Also, both groups of students were in a special program. 

The McAllen project was one of the six true experimental designs we hav- 

™«J D j£h U " raCU " and 33 such che "ported success should be given " 
greater weight than the results of studies that were less well designed 
T^S S«eralizability of the study is limited because the eSer - 

M™? a i, Childr ? n co °P leCed 0 ^ '^ndergarten and the experiment il 
limited to only one school district. 

The ongoing experiment in McAllen indicates that immersion can indeed 
(SSL Ch % Cypical bilingual setting, and that perhaps the 

Canadian experience does generalize more than had been previously thought. 
The difference between immersion projects and submersion cannot be over- 
emphasized . An immersion program does not involve simply placing lanzuaee- 

1~i^ ChUdre Vr " ^^P^aS cUssroom to' s'ink or Iwi^ im- 
mersion is a carefully structured program of L2 instruction which presumes 
no prior knowledge of L2. The teachers understand student queries InU 
even though they reply only in L2. Immersion programs may co meet cne 

:r S h! l, T lg ^ n ° rit7 children. They require much work on che part 
of schools, buc chere is a payoff for che effort. 
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CHAP7S2 3 
CTAPP1ICABLZ STUPES 



Tha preceding chapter discussed chose seniles we found eo be apolic- 

r^ qU89C i^ i Q , C8r3S ° f (1) Che iS4iUfta addressed by che scudies 
I?dli J?}f f cho ?° lo £ cal soundness. This chapter discusses che studies 
£ IS °" r±ad ,? P P^ 18 ' ?* discussion is limited to che reasons why 
*e did not consider chese studies relevant Co che issue. While many of che 
studies douocless address some other question(s) adequately, ve are not 
overtaking a general review of ch, licerature acrosl^ll oosslbS qu^s- 
c-ons. w« are accepting en answer some parcicular policy questions, and 
so are concerned only aich studies chat provide suitable daca for 'hese 
questions. 

The aosc common reasons for deciding chac a study was not applicable 
~r our purposes were the following, alone or in combination: 

o The study was designed Co answer locally relevanc 
questions chac did noc address our quescion. 

o :io control or comparison group was included. The 

study reported gains only for the program participants. 

o The norm-referenced model was used. 

o Ifo statistical or aatching controls were employed where 
assignment to the program was nonrandom. 

o A posttest-only design with nonrandom assignment was used. 

o Results were reported only in terms of grade-equivalent scores. 

bUl l! 0f , chl3 ^Pcer presents our reasons for rejecting a number 
or studies chac have been widely ciced as evidence co support craosicior.al 
bilingual education. Ac che end of che chapcer we describe an English as 
ajecond language scudy of a projecc widely ciced as evidence supporting 

Our StfV" 30 ? 8 ** rajeccin ? che not addressing our questions. 

Our final discussion presencs a summary chart of studies which did not mee* 
our methodological criteria and the reasons for rejection. 



Transitional 3ilingual Education Studies 

The Chiapas, Mexico, Study 

Description 

Modiano's (1968, 1973) comparison of the Spanish direct teaching and che 
Indian nacive language approaches in che Chiapas highlands of Mexico is 
profc 'y che aosc frequently quoted study in che area of bilingual educacion. 



1 



1 1 7 



*££ «1e^h f 3CUd7 h " ******** * (1975), M quoce 

The Chiapas educacional situation offers a natural experiment 

Sl^"r Ch8 L haV€ devSl0pe<1 Chree W o£ "hools. Two use a 
direct "aching aechod entirely in Spanish; these are the Stace 
and Federal schools. The chird is run by che Inscituco Macionai 

EfSSEL 22 * r 1 icCroducM i^^ruccion ia che vernacular, 
xae children begin in a preperacory grade. During chis year 
-hey axe caughc reading in che vernacular using a global sechod 

w "Zfi"? or4i 4 S P«4* h d ^^' In che second year of school- 
.ag, children are incroduced co reading in Soanish, chis cine bv 
a pnonics mechod. In Modiano's scudy wo factors were exaoined': 
che effects or che cvo aechoos of icscruccion on learning co read 
in Spenisa, and che role of che ceacher in che ccmmunicy. 

The sample was i,601 children from 26 ichools wich 42 ceachers 
.n chree cribal areas. Villages (generally wich one school 
each) were macched as nearly as possible on variables such as 

22*25 i ZOm V°f d ' aa ° UOC of food ' clioaCe of ch « schoolroom, 
and resistance [coj or accepcance of schooling. Careful obser- 
vations were made of actual classroom procedures. These obser- 
vations, concained in Modiano (1973), highlighc che importance 
or teacher training for rural teachers. In addition, her com- 
ments about poor attendance, low aorale on the pare of the 
teachers, lack of materials, and isolation of the villages 
underline the need for analyzing the ecological situation of 
che school for pocencial alternative explanations. 

I^.^^'vff'fT'"' a Spanl3h ^S^S* wading test, was 
developed specifically for this area; items were made relevant 
to the Chiapas Indian child. Reliability and validity estimates • 
were adequate. The test was given to all children who the 
teachers considered knew Spanish well enough to take the test 
(about 3C percent), the children were not eouated on age or 10 
since it was virtually impossible to obtain chac information. 

Modiano (1968) compared che DTI schools and che Scace and F-deral 
schools on a number of variables. The UI school ceachers ider.c<- 
f-ed signiiicancly more children whom they thought could -ead " 
Spanish with some understanding. Theii- judgments were corrobor- 
ated by che finding chac che DTI studencs also scored sixni-'i- 
candy higher on the Spanish reading test. 

Reasons for Rejection 

• There are several major reasons for rejecting Modiano's study. ?<rst 
and most important, we have reason to believe the comparison group was "not' 

either che Indian or Scace schools was noc random. Variables known co af- 

lear ? lo? WBre not "acisdcally concrolled. Modianc acknowl- 
edges she was unable co coccrol for age differences. 
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«„ u^° Ch ! r I*?*** Q0C controlling for prefacing differences l'es 

la Modiano's acknowledge*™: that there was higher adult literal 2 \t 

5"2E KJ* 8 "' J 0 * 11 * 110 «*■* the literacy race w*f c £ «s«lc 

or adults having participated in che native language program However be- 

ties h£'hlrii??Jii! alteraac± ™ "Potion would be that these communi- 
ties nad higher literacy rates even before the advent of the Indian schools 

aiue^r 5 f \ ° concro:LL * d - ^«»i«o seeas -.0 have made ao statistical 

adjustaent ror chese preexisting differences. 3C * caA 

-± on tfw C S r w! r °J lM ia " odiano ' s 3Cud 7 H «!8h the method of L2 instruc- 
tion utilized by the Federal schools, which Engle describes: 

The Direct Method Approach advocates fluent secocd language - 
learning before reading is introduced. Modiano's (1973) da- N ~- 
^f!^ 10 ^ ? ^ S?a " ?ederiL ««J«>ols. which were u*ing 

?^?J^J , / ndiCaC !f ChaC Ch ° were far from fluent 

in Spenisa before reading instruction was begun. Further no 
systematic oral Spanish instruction was attempted. Children 
learned by rote means and were generally confused. Thus, he- 
results suggesting chat che twelve language approach in DTI 
schools [was J superior are not surprising. The study does not 
Present a comparison of a good use of che Direct .Method with 
the Native Language Approach. The variable of che teacher—his 
background, ethnic identification, training, and relationship 
*«h the community-has been coo infrequently studied. Modiano 
(1973) assessed che relacive effects of the native teachers 
(promocores) and che State and Federal teachers, primarily from 

?S™ r°f^ n ^ CttUure ' on ch « c°™±cy in which the school was 
located. Promotores are usually sixth grade graduates from an 
Indian community who receive training-. She used chree measures 
or che expect or ceachers on che community: che number of 
teachar-sponaored projecca (chis is one of che roles che teacher 
is supposed to fulfill); che percent of girls ^ school; and che 
percenc of females vhich che ceachers indicaced knew enough 
Spanish to be cesced. Chi-square analyses on each variable 
between che two kinds of school were significant, favoring INI 
schools. Indians also unanimously preferred Indian teachers. 
Thus, ic appears clear chac eicher che Indian ceachers, or che 
vernacular in che schools, has some effect on the rest of -he 
community, and that this effect is greater than the effect "of 
the mestizo (non-Indian) teacher on the community. 

Is che difference between che schools a function of che lansuaze 
method, or of a sensicivicy co che culcure che Indian ceacher 
brings chac a mestizo teacher would not have? Two Indians in 
her sample ceach in che Direcc Approach schools. The children 
learned more from che cwo Indians Chan from che mescizos, buc 
less chan was learned by children in Ul schools. A scudy or- 
chis quescion is needed. y 
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The Rock Point Navajo School Pro gran 



Descriation 



r.,*,,-^* ^ L± ^ U * 1 « cha Rock Point Navajo school has been widely 

32*.^ f°n« r * 19 J 8> ; FonowlQ S cl » successful develoomentVthV early 
*2L Jf s in E6rfllah for N * v ^° «udencs ac Rock PoL" 

SUfiSi? \^ngual program. Initially funded by TLzU l, 

Uaiced co ««Sina«r (?re-first grade) W 
Children continued to receive concentrated oral English inftSction using 

i»SS- a ?r°? eh ^ i 11151 * 1 Uc «^~«2din i _reaSn«s-- W as introduce *n 
2S%n 1976 ' ? * ^TTT^Hnis-added). £ 197?" 

"a^owS t£*! fcT^ Ch T bUi3gU * 1 ?r0ffraa Co Srades S and 

allowed the school to develop a comprehensive bilingual curriculum The 
™ «x?«nd.d from just reading readiness in £SjJ°£ c£p?ete 
initial literacy in Navajo. After the children learned to read well in 
Navajo, they were introduced to English reading (during Sade 2 or 3r 

S r r esuSJ" Rlla ^ 1976 ' P * 380) - * ^"^Sngua! eouca- 

tion resulted for those children who had graduated from the kiSariarten 
program and completed the first grade before 1971. *^rgarten 

The evaluation is based on three different analyses. First it cm- 

P^oSam^tr» nr0lle i in 3T8dM 2 Chr ° US * 6 * che Poin biiingSl 

11??™ ; ° CCap * ri5 ° n3 S™? 8 draw ° other Navajo schools. Rock 

^^sin^-sr 1 * eh cfae SAI 13 1970 ' i975 ' i976 > - d 1977 ^ 

St ; u ^ 1976 * 008 comparison group was tested in 1975 with the 
411(1 cil « other wts tested in 1976 with the MAT. 

inst^crinn conclude chac Mav ajo students who received bilingual 

stS! ,r ° n scand * rdi " d achievement tests than did Navf j7 

students at comparable schools who had received English-*nly insrruct^on 

£2 £*\ coac } ud « Chac chese 3Cud «" d " better than an eirSr^otott of 
Rock Point students who had received instruction only in English. 

Second, a comparison is made between Rock Point students in the bi- 
lingual program and Rock Point students before the program began and i- <« 
reported chat the bilingual students score better than the eaKLr grouV 



who rh^^f 13011 ! 13 beWMn ona « roup of fourth-grade students 

oT WhlrSlJ fZTtl ?r0gra ° C ° nCinU0U3l 7 Jcindergarten and a group 

or .ourth graders from the same year who had had their bilingual program 
sequence interrupted in grades 1 through 2. It was found tSt^he stents 
with continuous bilingual education performed better than the group Sose 
bilingual instruction was interrupted. 



Reasons for Rejection 



One problem plagued all three analyses. Each analysis vas 
conducted in grade-equivalent scores. V« aoted in our first section the 
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unsuitability of grade-equl-ralant scores ta orogram evaluation- i a <* 

'tVZ^Tr S 3 300(1 of oae of" : ££2! t 5£ 

Jm Jock Point students were tested with both the SAT and the SAT. G-owch 
rates in grade-equivalent scores were calculated for both tests z* 

2S^*£i2S" cha7 3hmad 3how ch * both 31302 " 

(Ssur,^* 2r ° WCh ra " ™ laS ° n = he ^ and 0.« on the MAX 

stucw! -S:! 6 -^ 175 - 3 ** lc ? COT P arad ci « Point bilingual orogram 

lll^ll I groups rroa other Navajo schools had serious lethodologica^ 
problems, Rosier and Hbia tried to match the treatment and comparison 

d0U i" ?• CO °P* rtsoa *<*°°^ similar cTch. Rock 

^ . ? 7 : ^ ada<luac7 0£ " ch « comparison batween Rock Point 

G T ndS ° n ***** a thelSaSent and 

•STSS^S"^ ^ ia v (1 T» P ' U) re P or " cha comparison scSoL 
noils Z£lt ° C ?" r NaVaj ° 3ch ° 015 ««hod- b« 

SS^i. W 0al7 * f8V 3IA 3ch00la "^programs.- Further 

fSa uffS^ ? Che a ^ools receded students « ^aes 

3 and up from .ederal scnools who,, instructional programs were not kaowo. 

h*« M^J^f*"' Ro3iar and 3010 (1980) chat the Rock Point school 

c^tS^I 17 ouc ?«== ora « d octar Savajo schools. They showj that S 

chJs fi J" SC °" d hi3 ? 8r Chaa 0Ch ' r lftd4 « s <*° ols «■ interpreted 

™S ? a P 0816 ^ 8 0UCCOT « -^r participation in tha bilingual 

Sf r \ I 0 ^ 17 ! aC ChiS hosier and Llm should hWstowT 

Ss £»nIS^ aCd / he "f" 01 &di * 1 3Ch00ls «•» historically IJuJvSnt. 
This demonstration is particularly critical in view of Rock "oint 's hlscar^ 
cal superior cv CWUlink, 1963). However, Rosier «d hX^£«2S e£er -o" 
test for equivalence or to apply statistical adjustments for differences. 

-7 oSr^cS ^«™, U1U3 ^ aCaS ?tinC ' S ** sucerior- 

feSTLSfStTMolS "2 3IA 3Cfc ° 013 - 111 Ch " CaWe w * «trac=ed':he 
stSnrfhff^ i 9 ° ""P* 3 * 3011 «*oup) and the performance of Rock Point 
VocfSa * he ° rQ?raa " a52^£2£ed from Rosier and Holm's chart 12? 
si , C ca * pattern that the authoTs interpret as evidence of the 

S£» S^r " P"^r- in che 19?5 data is fo^d^l^O, 

the cL^r^ b * gan * * b " h 7MrS ' toek ?oinc stud «« began below 
the comparison group at grade 2 and were above by grade 6.* 



^rif^I?/ 1 " °^ Che si ^^rade differences be«ee n 1970 and 1975 

S*"tta iSSTaS, 4 ?rograa affecc * aowaver ' ocher ?033iile « a3 °- 



o 
o 

o 



Different cti«ca w«r« used in 1970 acd 1975 # 
Dlffarenc comparison groups ware used. 

^Kr" C f e ^^^•^qviivalenc scores cukes such a comparison of 
doubtful validity. 

The sis, of the difference is less than the year-to-year differences 
soaet^ses found between consecutive classes at che same grade. 
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TA3L2 3-1. ROOC P0DTT AND COMPARISON SCHOOL PERFORMANCE 
, EI GRADE-EQU IVAL2NT SCORES ' 

;;- m 1970 197 S— 1077. 

Grade RockPoinc Conrcarison ut^ ence , aodt P B <nr cJarison M » W1 ^ 



2 


1.8 


1.9 


- .1 


1.94 


2.16 


- .22 


4 


3.9 


3.2 


.7 


2.67 


3.13 


.46 


6 


5.4 


4.4 


1.0 


5.2 


4.15 


2.04 



n .Z 111411 ^' B0Ca the daca Rosier and Holm used co deaonscrace chei- 

ssstjsi i co ° pia "- 1375 *>* pome 3isC h ««3rsrsL2Sd = " 

o«cau34 char* wtra coo few snin4#nt-« cat* ^ 

r««rsv4 /^-t^v 3t «««ncs {six) . The SAX comparison group was 

cescmg program (1975-1977) co chis one comparison case year. 

' oro*r« ^* B 3 * cond ^7?^. grade 6 students in the Rock Point bilingual 
V T ^ CMC8d wldl « eh « I 973 SAT and t» r « compared with earlier 0970^ 
lock Point students who had been tested with the edltSn of cnl Stf The 

JSTS/iS? Pr0graa W,C8 f0UOd C0 154 superior! but again 

Jr'ff' difficulties with this analysis. It is not at all dear that 

u'^alo^lr 9 ^^^^" lndeed " C8ive realing e *struttion 

the^IorL' ,l\ ^° a °l Wh " cakia * ^renc cescs had on 

the scores or how successful published cables equating grade-equivalent 
scores across differenc edicions 0/ a test really are! StMe ^ uivalenc 

similar?!, 0 * Seri ° US P robl3as is the lack of analysis comparing 

« C0 5°? S b8W,,en Ch8 pr0grao and 1970 cohor =»« SeSous prob- 
iSoTLa T Paring " h °" 3 <* »^d«ts within the' same 

££! ^ ^ C ?" 7 !" ^^enc when the number of students is 
2Si\ ^ Wri53 0nl7 " y0U 030 4331106 each succeeding cohort of stu- 
dents is the same. Although the law of large numbers sals thi"is the c aS a 

III Zh l T iCe P 0 ^ 1 ™' everything else being equal! one or cwj schools 
a b r L n ^*«an e cL i 7h P a P bi1 a S 0n */ nd f"" ^"-Sous 'caTpr^duIe otTidlr- 
classes SS^Sr f "JfT *° d p « foraaace »»7 wo successive 

classes. This poinds illustrated in the Rock Point data, where two suc- 

5? eh ^ (19?6 and 1977) had ave "8* 3C °«s oi 5 66 InZ I £ 
about a 25-pe.-cent difference in performance from 1 year to che St t ex- 

SKISJe '"XL 36 "!? indiCate9 a V * riaCloa in ch " numSr of SSdren 
found C *!£ 4, ?*■ US""? 1 Can aCC0 ^ C ' for cohorc differences being 
aaSms ^ n * C ° C ° nCro1 f0r Cohort differences in their 



becwJn 2* ^ TSis, Rosier and Holm (1980) describe a comparison 

5.£lf«f£ * !! ° £ , fc,uK:h 8r ad «". lowing che results of continuous ° 
2£?, i" e T PC f 1 ^JS"* 1 instruction. The interrupted group begin 

J ?2 j ° r ! adiDg readine " instruction (apparently^ in kindergar- 
ten) but then encered che all-English firsc-grade program. In chTcES 
grade they M recurned co che bilingual program/ Rosier and Slot argue chac 



122 



S"!,S C * P 10 ^ 8 38 iaportanc evaluation or cha program. They accribuc- 

r££t£.^ ^^i?rosrm. There is anochar iacerprecacion co "chase " 

HZtu ^ ?° W * Ver ; Th,M Sh0uld b * Xiccla WOnd « chat children who began 
schooling in one language,- we re changed to a second language and then 

do so well aa-scudancs who had_a_coordinacad exoosure -co chi wo- lanwn, 
curther, Rosier and Holm failed co examine equivalency becSen the ST" 



Finally, Rock Point is but one o£ three .American studies to use the 
E2??£*? E4 " C Ceachia * U literacy and then teaching L2 licaracy. the 
f!f^ C * Ciag ,* r f ,Ce f U ch4C *» ^Pocnasized by first teaching Viicar"- 
If. ?• Mj0t - 4uacifi «tions for T3E. Consequently, Rock PeinT 

™ 5 i i» doubttul aince not all the classes in the bilingual program 

alters S Ve H b€8 ° 5 lrS , C CaUghC UC * taC3r in U ' 008 °* «*• 

aspects of the study focuses on how grades 4, 5, and 6 were cauahc initial 

? C S?7% ?° Si I r ^ 3010 U980) Sta " Chat cha f "<* Sd ,5Sh LaSrs^ere 
caught initial literacy in Savajo in grade. 1. 3 ut if the dates £nortld 

other than kindergarten reading readiness prior co 1971. 3y the 1975 res t- 

? chac*^* "S ^ gTaderS ln 1971 Would ^bJS £ grant 5, 

ttal^tafJv'i^" f ■ d */ 0nly ° ne Would h ™ been taughfini- 

tial literacy in Navajo and of the three grade 5 classes, only two would 

sc7red M tL C ££ C l a SinCe iC ^ 5 ^ 6 cLbL w^ 

chTrtMul" Stte^eteJ.^ 11300 ^ " » ~ "~ h0W 



Finnish Immigrants in Sweden Study 

Description 

sv«~. r ?c °J 3 ^ ° f Finnl3h "S"" children in wo Swedish school 
systems (Skucnabb-Kangas and Toukomaa, 1976) is fast becoming one of the 
mat widely cited studies supporting the use of U in the schools (see 
Piter, 1979; Troika, 1978; McCounell, 1980 a, 1980b; Baral! 19^ Rodriguez- 
Brown and Junfer, 1979; Cummins, 1980). ' !ia<irt ^ iez 

(1976? C ,ntwf*r eral S Cb ° UghC u Ch *f. Ch * ^ CUd7 b7 Skutnabb-Sangas and Toukomaa 
LSL^T 2? ?' n0M 3chooUn S in F i^h (Ll) children had before 
beginning instruction in Swedish, the better their Swedish (L2). It is 

f" i?f arred J: haC chis findin 8 supports the use of Ll in the Oniced States 
for children from non-Euglish-speaking backgrounds. 

Reasons f or Rejection 

^£j* j ' Cted Zh L Skucnabb - K »°S*« «d Toukomaa (1976) report for several 
important reasons. The report lacks much of the decail needed co figure ouc 
exactly whac che researchers did. In addicion, the auchors did no c have 



if!!**"' ^ Ch " 7 did a0C «y co ««* «=he comparison and exoeri- 
aancal groups or so control statistically for oreexisc'n* di-er^f 
Further _ caere is a high attrition rat. In the- auSrs'lam^? 

the *S2J la L d *!? * P*" 1 ^^ "vera problem. The authors state that 
the study covered 637 students (Skucnabb-Xaugas and Toukomaa W76 T 48) 

-cent ox tha^S al7,i ^ i A^ ed ° n 0nl7 150 S5 alaLt 80 per-' 

thTsclv ai3 l ing ' 3ari0US <*«°"ions can be introduced into 

the study. The authors give no consideration to this problem. 

of J^T^T* aad J T0uk0o * a Presented no statistical analysis of many 
clu de^ n !Sr°fh^M d * nC ! % SW<den - Sfca«*bb-Kang M and Toukomaa (1976) con- 

s^rvec COagUe *' 13 ° Cher Words ' Che fce « er » ?upil has pre- 

served cheir mother tongue compared with others who have lived aTaoual 

length of time in the r*e»-f*rf ► . iivea an equal 

sieea for i™ r „ ""iying country, che better are their prerequi- 
197o! p! 78^ 8 f0 " iSn Un S ua 8«" (Skucnabb-Xangas and ToukSaa, 

•<rashen SST ^ 1" " *»«a=ive interpretations, 

•s-asnen in a recent review of the literacure on a<r» i ? i — 

lag, concluded chat older learners acquire the Sc^a! sSeffx^ 1 

taSS'jS" d ° 70Uager leara " S ' -t^ungetlea^S "S^cSylea^ 
higher perrormance levels. Krashen's findings lead co a verv differed 

Se^Y'n' C ? 6 educacioQal W for lsng^ge-minoriVscuaents. 
iTto usS £ P £ Ca h C e ° C f" b " n draWn ^ Stnabb-Kangal an I Toutomaa 
i! Che aarl7 7 * ars of schooling, the implication from iashen 

on £."2?** 1136 "M 2 in Che aarl ? t*aguage educator "22? 

P re? ° rCad b7 :<raahea have "ressed the iaporcancrof SSnSiaa 
second-language inscruction at the earliest possible age. ^niung 

-s. J C * 0uld bC necessary to control for both age and U proficiency before 
the implication chat language minorities are besc taught fiwcln 2 l^T 
be demonstrated. Skucnabb-Kangas and Toukomaa did noc do chi" 

J.^ " 2 .. T !! t / e !: 0 r aanCe Snd Sch00llng in L1 / Perhaps che mosc imporcant 
SbL ? f tut 2* bb -^««« *nd Toukomaa 's report on chose reproduced here in 
table 3-2. These data have been widely interpreted as showinTthac L2 Sr- 

£cT? Ce r iS beCC8r Ch * e lon 8« ch « «Poswe co U: third to sixth Laders 
with 3 or more years of school in Finland performed beccer in Swedish rhL 
did chose wich 1 co 2 year, of school in fLland who i" cu?n Sd lecS? 
than those schooled entirely in L2. (Points refers to cLee^tSo-ils 
of perrormance in Swedish, wich 1-2 being che low end of 1" S2" 
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TABLE 3-2. RESULTS OF 5JRITTB3 COMPB5HEISIOM TESTS ACCORD XX TO 
LOCATION OF SCHOOL (JAUHO AND LOEKANEN 1974) 



Percancage 
Points 


Attended school only in Sweden 


Attended 


school in Finland 


sweoisn-lang. 
class 


Finnish-lang. 
class 


1-2 7rs. 


3 yrs. or mora 


1-2 (-) 


122 


52 


142 


62 


3 


262 


402 


112 


122 


4-5 (+) 


622 


552 


752 


822 


Tocal 


1002 


1002 


1002 


1002 


N 


65 


40 


28 


17 



SOURCE: Skucnabb-Kangas and Toukomaa, 1976. 

authors present no statistical analysis of the data. We carried 

S^rl St? these data support the use of LI instruction vith language- 
SSf *L 411417313 13 ^ two factors. In=e¥^ 

oreters or Skucnabb-Xangas and Toukomaa have assumed tvo different under- 
lying models of the language learning process. Some models assume Tcon- 

tonZS^l*? ^ CC i°r" Che l0nger Che ?inai3h SJSw educated 
SwedTs^T^M^ i h3i u r 4CadeaiC achi ^nt was in courses caught in 

ISaf 197?; n ShS 8ar i n ? h ° U3 * f ° r 3mnSU41 Sducac i°n, in Skucnabb- 
S^ihSL* « /!££* " SUM * ^^oneiaaous, step function; Cummins' 
threshold hypothesis (1976) argues chat a certain level of LI skills muse 

SliSSSS f V faClliCaCin « « fecc ° f U on L2 will occur TteTp- 
SJiSj^iF 0£ J V *FL 0M "atistical tests to che data depends in part on 
whether the underlying function is continuous or has a step. "ae steo func- 

natLS^i* d t Cfa0C0 ° 7 in Ch8 and «"«l*ng function and the use" ' 

categorical level tests. The continuous function implies chat the chrte 

FinSni 0t 1"' 3Ch00l i ? L WSden ' 1 C ° 2 in Finiand . and S yelrs S 

apP^^Ita? " ""^ "* " 3 " 3Ui - C4ble for ordiaal a " 

ceiw c s ;rSh?r pl ? cacing fa f or 13 chac cha 30211 sanpie ( °> 

, * rai3M < ^ uesciona about the suitability of che?T 2 test. 

\ our analysis will include**, likelihood ritioTc" asymmetric 

standi ?.Lt tUT* V4 i UM * (The <^ed by che asy^totic 

cilcalfSd confident.) Table 3-3 presents che scatistics 
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TABLE 3-3. SUMMASI OF THE STATISTICAL ANALYSIS 



Test 


1 

! 


2 


3 


Analysis 

4 5 


6 


7 


8 




11.2 


4.79 


2.59 


2.65 


3.49 


2.59 


7.7 


3.08 


Pro ha hi 7 f rv 


.08 


.31 


.27 


.26 


.17 


.11 


.02 


.21 


UJcalihood ratio 
racio2 


U.7 


5.11 


2.33 


2.94 


3.66 


2.83 


3.3 


3.13 


Probability 


.07 


.26 


.24 


.23 


.16 


.09 


.02 


.20 


0.0 


0.0 


0.0 


0.0 


0.0 




0.0 


0.0 


Somen's d 


.09 


.14 


.21 


.196 


.15 


.21 


.140 


-.03 


Approxiaace 
probability 
less Chan 


NSD 


NSD 


NSD 


NSD 


.05 


.05 


NSD 


MA 


G Annua 


.136 


.29 


.46 


.437 


.36 


.49 


.276 


-.05. 


Approxiaace 
probability 
less Chan 


NSD 


NSD 


NSD 


NSD 


.05 


NSD 


NSD 


HA 



Noce: NSD - No significant difference 



Each coluim or cable 3-3 presents the results of an analysis corres- 
ponding to one possible interpretation of the data, the SSKllS^l- 
fot Sonets'. id and gamma are approximations to the t-distribution blsed In 

ftatSS'sS^ 3 !: 11 ^ "VS* * ^ ° Pi3i0n ' S^rall" the iost^exu! 
IT'IIT 5? ?! coefficienc . gives the probabilicy of knowit 

""8°^ °* Swedish performance a scudent will be found «SZr3. 
scudenc's exposure co school in Finland. * glven - ae 

Our analysis is as follows: 

1. The first column in cable 3-3 presencs che analysis of all che 

bSVS 0 ! T anJ'L 3 ! 2 - * a J di "° n " ChB "^^tics-Jresented inla- 
The orl iiTr fpearaan rank-order correlacion coefficiencs were compuced. 
The probabilicy value of all chree fell wichin che range 0.10 + 0.02 The 

T J?*" Claar * N ° MCt8r assumpcions are made aW che 

mecric and che appropriace cast, chers is no relationship bec^enLi and 
L2 performance co be found in che overall daca. 
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<n sJd.n £ ? l ^ « cludia S ch « Sroup la Che bilingual program 

; ^* theoretical reason for dropping these scudeacsfroa the 
analysis is that this group introduces another variable (bilingual educa- 
tion into the analysis. Also, since this is the worst scortnTlroul 
eliminating it maximizes the opportunity to confirm the SaSuloa! 'as 
cam o. seen in 5& (2) of cable 3-3, the bacc*ry of uouparamecric cists 

£SLE»£&"™ e a J SOCiaCi011 * ci » da"' * * cesc, been che 

S!!rt. S * ^ a oae - w *y analysis of variance' were oerrormed. 
These scacisc-ics were insignificant (P<0.4 in boch cases). To further ex- 
plore the possibility of associacion between che cwo variables, Se Pea-Lu 

AiXSZJ 3 *^/' "fT***"*- coefiiciancs were calculated IT 
-*re insignticant and in the range 0.15 -r 0.02. 

oosure^lf iTt^ **** W " ° f Che ^^ng model chat orior ex- 

effiS 2 n a3 rf 1Ct ^ a 13 * c ° aCi ™°« runccion showing a facilitating 

HZFL. ~ "\ ^ 13 Q ° ia Skucnabb-Kangas and Toukomaa's 

daca co support such a conclusion. 3 

nn rh.I^ l ^^ ^, Chi3 ^y 513 is Probably che besc cest of che chreshoid hy- 
pochesis since none of che proponencs of chac hypochesis seems co identify 
a school 7«« where che chreshoid is found. When examining^ ?ull rangT 
tnrSSid 30 * 13 f0Und ChaC W0Uld cor " 3 P° ad » ch « Presence of 1^ 



3. 



. , Inspection of che daca suggests che chreshoid may have been oassed 

aSLSL 31:0110 * th 1 " 2 7ears °* schooling in Finland was 

encSetvln^wJin T ^» «» 3 c«,m«d co S. group- Soo! d 

years 4 of Shf^^'-f? 10 !' Che chrMfaold Hypochesis, sendees with 3+ 
iSfJi * rf^° , n fialand C0 «P ared «teh chose wich t co 2 years of 

school in Finland combined wich chose schooled only in Sweden. Oncf again 
there is no evidence of a chreshoid effect. * ' 

rh- -fl f 9 f h8±r discu « 8loo » Skucnabb-Kangas and Toukomaa (1976) refer co 
SnL US of / chl ««» of a °rmal Swedish pupils (4 or 5) " Following 
Suts in S ^ C3 ! biaed Ch ! Wo lowe3C 3ki11 ta^l categories (called 
SchocomLed compared che chree groups wich skill-level 



gana * < and Soa»»'a d showed small, but significanc effaces. How- 
aver, che use of chese ordinal amcric scaciscics wich a measure having oo£ 
cwo categories is problemacic. A more appropriaca M asu?e iTSwTSL is 
Tc. also significanc f c - 0.17(c 1.98, P<0.05). 



Toukomaa (Syf^^'f 00 ^"i* 1 ? mentioned by Skucnabb-Kangas and 
iouKomaa (1976, p. 66) is produced by eliminacing che group wich 1 co 2 

EE.*? ? Fia f ind fr0a Che ^ in A above In chis 

comparison only Somers's d was significanc. The aire appropriacef, was 
noc significanc (f 3 - 0.17, P>0.1). PP ° 1 3 3 
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sa ^ Jesse's jiJtiLr? t ca d 20 

SEALS' Pr '" aC8 ? 3 "^ c i°«hip. Even'if a «S3law wtadOB- 

£ver7thS: 7 co«SerL S *S 224 W Chan those schooled in Sweden. 
dataT cons:W8ra,i . «• rind no support for / C he authors' claim in che 

sun™!; J*^ 7 ' 'f 2 " chtt Stattaab b-Kangaa and Toukomaa study is cited in 
sISLh f Ungual program in comparison vich the students £ all- 
STg£S*" »igniiicant difference in the performance of the 

-w^TnSS^SX M^V** Ch f corr8l « ion is that ve 

j s interpret table 3-3. The aosc it seems to show is that 

a^JSi 3 ^ - ^'^^ tar ° f 

« T!. ^ analyses , one is simply overwhelmed by the lacko'rl- 

5i5rS«22! Md : *** °" iaal7Se3 ^ 0Uad ?°«ible siSfic2ce\Is1 
cSL STSSf« Che C T f 3CifflaC8 of cha scaadara error,%nd Srderiine 

Sl^S B g 1 S C !.2S} d * C " a " d 33 d ° UbC£ul) in Che "rength ol 
a u.!r7nn . 7 vari -*°l« can be assumed to be ordinal. The imoor-anc 



SlanJsS SnI ^ ^ nal 7s1"- Table 3-4 reproduces the third of 
absence ifa^^V^ 0 "' 3 , 04 J ° r analySM ' 1118 ^So" *°« Sat che 

^fultTthTir^ve^ S jeara old is 

3h r Ch * C Chi3 i3 leva" 3 ; L2° C than d St ? a r t e ! eaCed 

"ined by any other group. Readers are cautioned to keep in mind that rL 

SSrSiS* fT^" in C4bie 3 ~ A d ° 63 not necessarily^ S^kS 
a «£w^ « D Swedl3h « Sinc « the st udents who had lived longest in ^ 
Sweden (the 0-5-year column) had reached a plateau that the other «mLm.. 
wer 8 3Cill approaching, the higher correlatJotj for tne tner^™ lo- 
cates that until the plateau resulting from spending a long tiS L Sweden 
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TAJSL2 3-4. COLLATIONS WITH SWEDISH-UHGUAGS SKILLS P? T^IRD TO 
SIXTH GRADERS (CLOPSTSOM) 





Age on aoving co 


Sweden 






0-5 yr. 


6-8 yr. 


9-11 vr. 


partial corr. 
\> c-ae aeia constant ) 


Age 


.19 




.41** 




5e^ (a) 


.03 


.12 


-.23 




Length of residence 


.19 


.Sgxxx 


.31* 




Age on sieving 


-.10 




.00 


.23** 


Picture VO<*ahiilai*v 

(Finnish) 


.51*** 


.15 


.43** 


.41*** 


STnonTtas (Finnish) 


.22 


13 


.24 


.20** 


Vord Groups (Finnish) 


.39*** 


.01 


.43** 


.32*** 


General level of Finnish 


.37** 


.03 


.35** 


.33*** 


Observation speed 


.27* 


.02 


,55*** 


.29*** 


Addition 


.44*** 


.49*** 


.50*** 




Better language 
(Swedish) 


.05 


.69*** 




.03 



68 



48 



49 



io5 



Mote; xxx, xx, and x refer to significant coefficients. 
SOURCE: Skutnabb-Xangas and Toukonma, 1976. 

In 1979, the National Clearinghouse tor 3ili:igual Education ouhil-hoH 
a paper by Skutnabb-Xangas which refers co and suS^rt. „re il^ 
sive presentation in Skutnabb-Kangas and ToukomaHw"'! SuTt ifSar 
docuaenc is reproduced here as table 3-3 showing oral perfoSanJ in S 
(note chat cable 3-2 refers to written skills)/ P« r -°™«« in L2 

LI i?rel 1 ac2 I 1o ng J 0mP ^ h,Mi0n 3U ** eSC Chac lo ^^ ■ctaaUng in 

^ aoove *or table 3-3 on the data presented in nahl* 3-5 rh* i-„ 

taken as supporting the conclusions generally drawn from Skutnabb-Xangas 
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TA3LS 3-5. LISTENING COMPREHENSION (SWEDISH) 
AND THE LANGUAGE CF SCHOOL ENTRY 






Group 1 
Started school in Sweden 


1 Group 2 
Started school in 
Finland 
School years in 
Finland before 
emigration 


Teat Mark (Z) 


Swedish classes 


Finnish classes 




3 or acre 


1-2 (-) 


122 


■ « 


us 


122 


3 


502 


33Z 


173 


122 


4-5 (+) 


382 


63;; 


692 


762 


TOTAL 


1002 


100Z 


1002 


10 QZ 




82 


49 1 


29 


» 1 



SOURCE: Skutnabb-Kangas, 1979. 



TABLE 3-6. SUMMARY OF THE STATISTICAL ANALYSIS 



Test 


I 


2 


3 


Analysis 

4 5 


6 


7 




7\ 2 


20.3 


16.5 


6.4 


4.96 


1 13.8 


.853 


3.9 


3 

! 8.53 


Probability 


.002 


.002 


.04 


.08 


.001 


.003 


.14 


.014 


Likelihood 
ratio 2 


22.1 


17.3 


7.4 


5.77 


14.2 


8.7 


3.9 


8.77 


Probability 


.001 


.0023 


.02 


.06 


0 


.003 


.13 


1 .013 




•12 


.156 


0 


.017 


.31 


.36 


0 


.145 


Sobers 's d 


.223 


.234 


.25 


.066 


.31 


.56 


.019 


.222 


Approximate 
probability 
level 


.01 


.05 


NSD 


NSD 


.01 


.01 


NS 


.05 


Ga^ma 


.385 


.435 


.44 


.397 


.58 


1.43 


.04 


NA 


Approximate 
probability 
level 


.01 

1 


,05 


NSD 


NSD 


.01 

1 
l 


.01 


NSD f 

i 


NA 



14 



0 

ERJC 



130 



St9 -e££ ' S ^ C8 Ch * C f 6POrr dealC '* Ch ■«*«•» ^Ils and the 

1979 .eport s.eas to address oral skills. Further, the 1979 reoorT's 

subject or the same general problems already discussed for Che 1976 ™«- 
.ormation on the test used and how the analysis vas done. 

.^-u !i naU7 v ** n0Ced «* rUar . there is a serious problem in the scudv 
*Uh the number or scudencs. Skucnabb-Xangas (1979, cabS 2) repor-s a 
sample or 68/ immigrant scudencs buc che analysis shown in cable Si t 
basea on only 177 students. The absence of dlca on Tr* chan" chreetua-ers 

Webb Cou nty. Texas, Math Probata 



Description 

Trevino (1968, 1970) looked ac a bilingual program in a discr'c- sur- 
rounding Laredo, Texas, which is locaced orTche L£? border! T^olec- 

^ e i eaBnCa ! 7 1 3Ch00l; Che objecc <* che Program was co -aaS eSch 
SSLiSS Ja'S"* ^T*; ^^^^^aking children consented to 
participate in che project. In che firsc and second arades a bilingual 

wSloc S^a? J^": * Ch « ^ che'celSers 

*M not bilingual, buc because chere w«re wo chird-grade classes 'he 

naglish ceacher Caughc in English half che day. ^ 

h_ trevino (1968) examined che efface of reaching mach in che students' 

SS SlSg. A ^""V X ! 3 bilia *^ caughf scudencs was SjSS 
2f e SJ arU f cohort , of 3Cu ^ncs caughc in English (L2) ta che first 
tTr 22 fl"?' ^^r 13 ?« rfora » d «r all scudencs and for a sub- 
sec *ich had 3 years of school without retention. The pattern of results 
was the same for the. total sample as for che subeec. Of signal vses of 
STS ch «SP*nish-sp.aking scudencs in che biUn^lro^ram 
wich che Spanish-speaking scudencs who were caughc only in English fSr 

S. ESSLr" 3iSQificaaC * : ™ <* *■ biLngua/group? BasicaSy, 
no dS^ST^ gr ° UP '? 3 3UP<ri0r ° n Che "ithmecic reasfniaj subcescTwhll, 
no differences were demonstrated on the arithmetic fundamentals subcase 



Reasons for Rejection 



There are three major problems with Trevino 's study. First. Trevino 

asst^e^vT C ° aaCCh - her " eaCaenC W and cohort To 

assume equivalence ot cohorts within the same school ia questionable 

S^H 3 ^ d h ™ included ««istical controls for cohor? daSerlnces. 

£JTc£ PreeSs^in; S«?" 1M€ ?" ob «~ed nochJng 

"o^am^s P nS cSe fe " nCM beCW " n *" «" C ° h ° rt3 and Chac Ch ^ 
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^r* S *! 0Dd ?r ° blea ia70lv * s ^"nuLL inconsistencies in Trev'no's 

compared -rich Saglisa-spealciag students la ch .i r respective cohorts! 
A third problem with Trevlno's scudy la »h« '-in^-tn* m,.* ... - ,. . 
»P"Uaj children la » r.gujto Eo»il»h corricata la pr.rtoa. mr. T^rT 

In eh. Ulingual cohort's having »re total oath lJtt2cion*Sa 

of 2.^7?^ f? 3 «Plsnation to account for che suparior^rforSac. 
of the bilingually taught Anglo children but fails to r**J i Z 
count for the performance of the Spanish speakers as we?f 5 e JhJS *T 
explanations can accouuc for Trevino's rasultf StSut^vo^ng^r^ 6 



The Colorad o Statewide Evaluation Study 

Description 



assess'SL'ol (1980) 4ad report a statewide 

assessment of bilingual programs in Colorado for che 1979-an i 

The authors used data from all available slnool dlstrii g htstacTvhere 

SEr*™ h CreaC3 * nC ' P"^" is evidenced bTctasses shSJ 

addition, they establish a second, stricter success criterion of a *aia of 
at least seven NCES (one-third standard deviation). 1 eri ° n ° r a gaia of 

third*t^fS 7 ' Was d8flntd 48 a Pattest score wichia one- 

coSJ J *f !^ a * Tl * C ^° n ° f Che P 76 "" 3core ' s ° ^t some of the classes 

CsSIf ,T *T 8 ° f SUCCMS aCCUally ^ have «P«ienced LcHning perfo-manc 
(Since these students were initially low-scor±n<r , j 0 „w.- " p«=r.ormaac 

standard deviation coward che S3 of STSlStai^^ ™a one-chird 

range of scores.) distribution woiud cover a considerable 
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Goldsmith (1980) concludes thac -overall, 37 ? «rcanc of che prog-aa 
reported galas or nainc.nance of acadenic acbiavacenc. . . . - ^ 5 



Reasons for Rejection 



. ^ * r,j,cc ^ ** GoldsmiCh's Colorad. Jcace study because chair 
«£L^ a ° C Tr trCOn " Cht ?r0blaa found 13 * nora-referenced design ( 3 aa 

2Jr2\! } ; < T** e " nC8d °° del M3ua " chac «*• race of iap^ov" 
«nt of scudants in cha program would have been Che same as chac of the 
noraing group in ch. absence of Che special program. This aay \ 0 Til -*m 
with language-minority children. 

A second major problem is chac che scaciscical analysis does aoc sup- 
£5io£ £Z S£S' T -f^=al readers will appreciate chat sh. procedure^ 
leading co an 37-percent success rata cannot be taken as evidence of aro- 
S^T 8 " 3 ' fT ^S*"* 100 th. -an artlfacTS Sw? ? 

e2JS^? r^'i 0 ",** 1 Ch€ aUCh0rs ' in^ion of a loss of up to one- 
deviation in very low coring children as evidence of 

S UC CCS s • 



r«, IS " onC,chaical readers, the problem can be illustrated using some 

S Ejects S^S? a T Ple ° f no r ai3paaic aCuden " ch « Sustain- 

20 L (SaS). The percentile score distribution was divided into 

20 parts-categories of 5 percentile points each-end the pos t test per ceSile 

T ^ n ~ C * P«« nc ^ ««*>«7. Therafore, ? tne 

^IS2 of "udents troa fall co spring perc.ncile cacegories can be 

STSS; t£ a S^2 k 001 ? 0 "? a " 484111131 '^ Ch ^^^ing scudancs, 
* limic che example Co che four lowesc cacegories in che fall (1st chrouzh 
20th percentiles). Since Egan and Goldsmith counted losses of up to tS! 
as ao change, w. will count a drop of 1 percentile category So?fSl to 
spring as no change. When we apply Egan and Goldsmith's logic to our daca 

rasult eg at l - ff 3 r d8 f r ?? 3P<Cial «■ find basically che sane 

3S£ tecSE J* 60 ""*'* ^"^"ted as a program effect. This is 



IABLE 3-7. PERCENTAGE OF STUDENTS ' PERCENTILES CHANGING 
FROM FALL TO SPRING (READING) 



Fail Spring Percentile 

Percentile Loas No Change Gain No Change T~gHZ 

1-5 

6-10 
11-13 
16-20 



0 


36 


64 


100 


0 


51 


49 


100 


15 


33 


52 


35 


22 


32 


46 


78 
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!! £"? 6 f*° "" i Coldaaich,s logic .co scudte.es coc in special pro- 
5E%£ < f " u h * coccl « io « 6>«e regular schooling product imores- 

u lo r achi,vlc « »^«cs~100 perceec of chosVbel£ cheuS 
SrS? % 3hOW,d < no ch * n «« or * Since chis cable is based on regu- 

elror clmS.nT.f 3? ' Clil pr ? gr *« iC che measured 

ralulcsT SC ° rM C0Uld haV * * ccouo " d «ot Sgan and Goldsmith's 

aarv (SlS l: J h ' 1 5!K r f 1X ?, 0 £, ete rMUl "' "? eciill 7 «=he executive sub- 
^Jt^S^JSV* " leC5lv «- ?or «»■*•. che author states, 

— 75 percent o£ .kindergarten data. . .showed substantial nins 'n 

excess ox seven NCZS . " This happens to be the aost excreaTposlcSeleU 
£L£ *i ( f • ta/ «»^) cable. The corresponding f^re for 

second grade of the sane year is only 35 percent. 



The Santa Fe T S ew Mexico T 3ilicgual Program 

Description 

Leyba (1978) conducted a longitudinal study of a Spanish-Snzlish bi- 

SSf ll t s g ^ in ^? 1 " * *' M * 3Clc0 - Xhree^le^ntarr^chools oartic- 
Ht^t^ * £ 711 Pr0gra ° for 3rad " 1 chrou Sh 6- Participants" vere 
volunteered by parents. The Mexican-American students participating were 
bilingual although they were stronger m English than SlpaSn. AeJ" 
students also participated in the program. Ihree groups were used- a »o- 
pam longitudinal (continuous participation program! group?" lit' ? 
longitudinal group, and a nonrandom comparison group. 

The Santa Fe program reported chat the Title 711 students showed an 
increasing capabiUcy in English language skills and machen^cLs over c-e 

faSLlr 3CaC r? i^l? ' Mjorie7 0f Ca " S Ticle 711 Sencs oucpe"!' 
toraed che non-Title 711 scudencs in reading and machemacics. He also 11- 



Reasons for Re lection * 

Leyba did not control for possible preexisting differences due to ror- 
random selection. There is evidence that 'such difference ISsted? Se 
program scudencs had higher precesc scores chan che comparison group in 51 
of 63 cases. This scrongly implies che program participants were becct- 
students to begin with and therefore might have made greater g£es "ether 
chere was a program or noc. ae * 

*rt, MU t? ° f i Che inzl ^ aia 13 ba " d ° n comparing grade-equivalenc scores 
-ich nacional norms. For che reasons discussed in chapcer 1 we find this 

ce^cT'oTS^r 17313 ; H r V ^' iC add±tl0n ' ™ gains'were 3 

tested for significance by the t-test. Micety nonindenendent c-cescs were 

Mr s C i S C ;i l5 , tfere , 3i f ificanC ' 2 of Che 15 i= che ™** SrectJon! 
This statistical analysis is doubtful. The nonindepondence of the t-cests 
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SSS 82 ss-tf-ss ^-Jffsraars 

Sc. John Valley School Distri ct. Maine Bilingual Program 

(TroUc, i9 "; SSi r vS ^Si r^i! " * lingual program 

the St.'joha pSe^S*. ££^5 flS*" f° rdtr * ±Ch C4aada » 

Sdid Ua * bl " " ° bC4in c °P iM of °* cheae studies. 

uV^oT 1 ; 2?^ « ^ iTS.S^ 



1. AS (1975e) 
Bescription 



AS (1975e) presents the results of local evaluations of rh* mn„ 

Stt IlfaEsS nrf C h °S ^^i" 3 ' Usi n« =he locally developed 

«SirJ?a flSinTS^V ' °J « rade ^^«" * a nora-r„ferenced 
W fajin^ , program students were perforaing at around zrade 
PW« - a 3U P pleo * nCal analysis, AI2. found duJT3l£5 

prograa students outperroraed studants in regular classes. ^ 



Reason for Rejection 



loca } 1 3 r developed analyses vere all based on a aora-referenc-d 

bi^h^I!- u analysis presents no data that demonstrate coa- 

ST^LSSTt'S Pr °I raa ^ C0 ""* rt80a P««nts no statisti- 

cal adjustments for nonrandoa assignment, and uses grade-squivalent scores. 

group get better, suggesting cSt eh! progr^^r J^i^tfcaa^s 
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SLSTr 10 * 1=3 achie T eBen " ° ver Zi3SB '" l00ke d at cohorts longi- 

££. JrSA. ™ C UP in four seances, ««nt down in five 
S!^. < r «»"««* the same in seven cases. AIR 'a conclusion chat the 

results of a sea zroun «oc hectur . wxuu ^ ;Lai - cae 

s wp goc oec,:e r every year is not supported by the data. 



2. St. John Valley (1980) 
Description 



wo™ ^JS. ^ -V " C P" sen " **z* on the percentage of bilingual 
program students achieving specified curricular criteria. ^ 

S 811 "* 1 ' che Program students at che specified leamine zoals In 
addicion, posccesc daca are analyzed in terms of grade-eqSvalent storL 

seten^af^H SC ° re - TO ar0Und and^hlrSuSs are ore- 

sented as evidence of program success. are pre 



Reasons for Rejection 



These IS the criteria-referenced and norm-referenced analyses, 

m » ? valuable co che local school, cannot be used for 

ZLrZ 11 e , ha L e n ° Man3 ° f deca ==iaing how a control groun would 
score on these criteria , and, therefore, cannot judge program k^ccs 

Cnott*h ? 5S5rJr^! 8 f 3C0 5 8S 1 are iavalid «•* »an7 of che same reasons, ai- 
cnougti posccesc grade-equivalenc scores and 3 canines are shown ?«*. JZ 

Sr^U^H 'TdT C ° nCr01 3r ° UP - 13 c ^s1nce of^trol ^ " 
Che .acc chac che scudencs scored somewhac above che nacional average rll- 
aoc oe incarpreced as proof of program effectiveness. * * 

3. vailleux (1977 ) 
Oescriocion 

r h 3??, 80 " Wldel7 ciCed * che several* evaluacion reports on che Sc 
John valley program is chac of Veilleux ( 1977). This scuS purports Co 
show chac achievement: increased by comparing che tSUfS cheers c 7 
Tears of che program vich cesc scores of scudencs for che 3 years before 
Che program scarced operation. His findings vere chese- 

° JS'-SS*^ b ? f0 " biUn 8 ual P«J«ct implemencacion and during 
SLniS < proj * cc ^ ars 3how general scudenc achievemenc has 
improved since che incepcion of bilingual educacion for chose 
scudencs in che program. 
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!!° je ^ St f** Z achievemenc during the 7 years ox bllisgual 
education indicacas chac Tide 711 scudencs in general, are 
achieving at or above national norms in reading, math,' and" 

° Title^VTT M,^ f districts indicate chat 

»™«. bilingual education students are achieving higher 
scores on nationally standardized achievemenc tests than stu- 
dents who are not participating in the bilingual project. 

Reasons for Rejection 

There are a number ox problems with Veilleux's analysis. His first 

SSren°t n mecrit1r» Sf ' C0 ^ 3 ^""nt'statStics^from 
oixxarenc occrics rron diffftrtnc tests. The oreocoffram snndenr* 

5S C :t1S E"' ^ Che pi«iSS. cLE with t 

JiLfJ SHA tests. The preprogram data are presented as the perceataS of 
students below grade level. For the program students the metric ?! rL 

ssrsr^oSSi* below cha ^."SiS « 

xor^JiJ ~ J 2 f 7 °f Pf!? 10 ^ 30 "udents scored below the mean cutoff 

31346 lev ? X whlle che **i°**7 ox program students * e ll at or 
above- che .mean stanine score, the program was considered a success? 

a a t JK 3 *™" ^.f^P" 01 * analysis since ic is incorrscc co comoare 
LESLIE ? * wlde - band category. Since che scani^e 

scale divides che total -distribution ox scores inco only nine cacezorltc 
large numbers of scudencs who would fall below • .mX 2 ^h. 

S^S"? or 1 * rad *-«n ValenC Sr^e-level score win ba included in che 
ac-grade-level stanine (i.e., five). 

ble S-o^r^rJ 3 / 8111 !^ 11 ^ 6 P033ible anore valid comparison. Ta- 
ble 3-3 extracts grade-«quivalenc scores from several of Veillaux's cables 
presancing poscprogram daca. Since 17 of che 23 (almosc 80 percent sctlU 
presented are "below grade level,- it is clear cha^S shifS from he 
S 6 s"u:ieis MCriC M 3CaaiaeS d "" ic ^ che pe^cLn of pro- 

T^BLE 3-3. POSTPROGRAM TEST SCORES 



Grade 


Grade 
Level 


Reoorced Mean Grade- 


-aquivalenc 


4 


4.8 


4.3, 4.7, 5.0, 4.8, 


4.6, 4.8, 5.4, 4.9 


5 


5.8 


5.2, 5.4, 5.4, 5.3, 


5.4, 5.1 


6 


6.8 


5.4, 5.3, 5.8, 4.8. 


5.1, 5.3 


7 


7.8 


6.3, 6.2, 6.8 




8 


8.8 


MA 
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... „--°f aover ' c ^ P"P«gram data included thre« school districts while 
.ha preprogram data came from only one of the three. It is instruct!^ 
caa P are «*• c *« «ore» for that one district for grade 6 only Grade 
scores j* r . available for both periods and are shown SUble S £S 

J^i,° glC 1 C0OPar± ^ 3UCh SCOrM ' iC is ^ear the program ha? 
carmrul errects: students performed somewhat worse after 6 years of bilin- 

S£ lTTlTu a ^ " d ChCir 7 ^ earliS ^0 ban "t partS- 

pate in a bilingual program.* Finally, cohort matching fails to f utlv llr,- 

.ailed to provide evidence that the two groups were indeed comparable. 
TABLE 3-9. GRADE 6 TEST SCORES 



Preprogram Postprogram ' Mean . 

SAI Graae ~ SRA Qrade- 
- Equivalent Equivalent 



Math Concepts 6 .1 Reading 5.4 

Math Computation 6.2 Matf, t 1 

Math Application 5.6 Language * 5*3 

Social Studies 5.8 

Science , 5.3 / 



As h^en^-r^V^ 0 * i3 ta *« l ° n no »""f fenced comparisons. 
As has been indicated before, a norm-referenced design is cot aopropriate 
ror language-minority children. appropriate 

Veilleux then presents data taken from AIR (1975e) comparing' scores of 
districts. This analysis has the following problems: 
0 Posttest-onl7 design with nonrandom assignment. 
0 Use of grade-equivalents . 
0 No statistical tests of significance. 

0 The mean scores were "derived from quart ilea and medians, and 
are approximate. In other words, there is qo way of knowing 



Note that we do not necessaril 7 believe this conclusion. We are only 
illustrating bow the author's faulty analysis can be used to snow exactly 

!5!l^?r iCe 0i WtUlC he cUlaS - We Chink ch " «*• problems found Tn tlJ 
S 'tT ttat aothing, either pro or con, can be concluded 

Tr Jl rf C - 3 ^^ he ? r0graa * H0W6Ver ' « che " ad « disagrees with 

aegat1v:' X ev 0 iL 0 ce: eilieUX 3 ^ «» *— * ~« *i. 



WSJ." 90 "* 1 3C0 " S a " anr * ere aear che ^adencs' «rre« 

SeSSaSrssL*" froa one of cha ocher schoois a * aia * 

vSieyT to make a case for program success la sc. John 



Valley. 
4. Troike (1978) 



sruJL Ji ff } conducCe<i a nri* of che recent literature indudinz 

££ZL ^ U H bla ° n SC * JOha 7alley ' Me «■■" h ^ antic^atedlroS's 
"alysia or che program to closely parallel ours. However it ^h !^ 
The major discrepancy involved the use of a coSroll^u"' *" a ° C * 

: sc-i 3 2jJ*S^C l0n u° f Che P r °S raa ,8 iv « b7 Troike (1973) differs from de- 

are made ab B out V Sr Ci0nS > F ° r «^ le ' 

EwSTStil *E£ - T f T 01 ranti0Bl assi Sa«e« la St. John Valley. 
■Arh »ift- !?« ^ randomly selected bilingual-medium schools were matched 
with all English control schools having scudencs of comparable 2 LT 
socioeconomic status, and after 5 year! (1970-75) biSg^al1y! c Sin el stu- 

JS^Ik&f.S Sr^f • 3CUden " * concrollSoL in English 
t!T?J, t M ' aad aava co ° c inued cb remain ahead ever since - 

^TSt alJlhaT^ ?" Pr ° j8CC «^»«*«- ^ilSd'Sr 
juuf, axx states chat other than national norms, no other comparison 

?L"!! ^° U ° ^ idenclfi ^^ la this report covering 197™75 S i de- 
SSS e J e (JS) C0B S 0l ch. gr S P v 0 ^^ 68 ^ ln -'of the'sctJof 

m Pa "ntro ^oS&^'ThS&^^S 011 """^ 1975 " 80 ' Che " 
4* 4* ^ , stoup oaca. Tfterefore, when looking across all che report* 

<a% of nonttol Cher8 W - S 8i£her raad0a aS3l ^ aC « coat LTter 

any rind of a control group ror most of che 10 years covered by the Reject. 

We also quescion Troike 's incerprecacion of che resulcs In nrMl ,„ t 

areas. I C is noC clear this ^ ^ cakan indicacive - a ' 7" f . 
feet, as tne following illuscration from the AIR daca delScracesT 

vich 2lt P S f S B ST 4 ° f 1974 C ° nCr01 ^ in che *• compared 

cnree S** aea -»re 0*2 gradetsqulvalencs below jrrade level t^rh „ n 
average standard Aviation of 0.9 grade-equivalencsf'tereforef che concrol 



So« a i C n S ;h7 ri ° r k°/ 9 h 70, SCadiM ° ada 137 3 uidan « «aff and supe, 
visors in che area had shown chac scudencs in che schools were perform- 

7alie7 (1MM — - 
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£°X."£2!L* e i ^ 3aCi0Qal n0Ea " " ab0UC Che 3aae levei a* Participants 
chosVL S a pr C 0 %S K0raanC8 * SCUdan " a0C *» <*• W« as .ell as 

je cannot account for the differences be ween the studies Troika ra- 
2l£ S stulv : , 8 ° W T r ' "* haV « — "nfidence 2 SfSlStej 

both rhl T*r?I S-t Che fiaai re P° rc closely checked for accuracy by 

ootn the Title VII program and the local school. c C u* acy o 7 

in St^JohTvS- ■? 3U " WhaC C ° iaca ^« about vhat hanpened 

be saoM^JSi!I\ ^* SCUd ^ brl£fl7 de3 « ibed °7 Troike would seei to 
se strong evidence in favor of program success. Unfortunately we have 

ELST-I"* ? fiCd 3UCh * 3iud7 ' Ba3ad ° n «*• wo 5^" sundry Ual- 
MEioJj ™ Project have been achieving at right around the 

T^r£SMS^ 13 giVen C ° 3usgesc chis ^ 311 a "-P^^nt 

Houston. Texas. Bilingual Program 

Description 

durtnfL (1 ?Sh 25*3 ° n T ° n ' 3 TiCla 711 biiin ^ education program 
during its fi_h year or operation and presents data for 7 ears 3 through 5. 

«an f «^ C £ Ugfa c federa117 f,jaded in P art - che Program receives most of i-s 
™ 6 C S^r^, rChea3Cera P 3 " 3 of che and ^ve a student enroll- 

7S*?£Z i«T2SSr^' 42 percenc black - and 5 

bioc e Li e io a Sat srs: L id dev^r cs 

t^n is 2ln2iS d * Un 5 Ua ? e 2 1 ' ^ Che Waaiad « the L7 Ltr,c- 
c-on is in tnglish for English-dominant and bilingual students while TnZ 

T^T^ SpaQi3h 3peakerS " cai ™ adMSonal subject matter 
instruction in Spanish after the lessons are presented in 'English? 

Matched students from eight elementary schools with bilingual programs 

tion ^TrTyefrs 'SfSL*" T 3Ch °° l3 WlCh ° UC 3 P"«Ste^S" 
™JTL,.h ^ ae difterence in gains between the wo groups was 

tested with no pretest control included. However, 11 c* ' 4 pretests ^--^r- 
ences favored the- comparison group. Twelve of 14 tests < II JJ^Jm 

5e W ^ 0 gr1fstud«S: ^ ° f SlShC " S " ° f -* gala3 Svored 
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Reasons for Re lection 



can-e^SS/ 3UCcfaia * w actempted, we reject the study be- 

S Sell ( J a ° Cad Ch8 £oUowin * liaitationa in the study, which 

o Students were eliminacad from the comparison groups ia order 

l^ 11 ^ P" CMC »cor«s. This marching process introduced 
unknown biases into the data. • 

o The program had high attrition rates. 

o All Spanish-dominant students in* the .comparison group were 

discovered they had been receiving SSL. 

these last two problems rendar the matching procass invalid SttuW. 
matched during the first .year of the study? but of the ul firsSrSe 21- 

l:T a tl ^ T* r ° f Cht "*»r/only 38 were SJe'te STfcLS^ 

3y «^"Cing ch. Spanish-dominant -students from the control «w 

pcr^anTb^inTth: ctpa^n! ' ^^.^^SSr^lIir 

t a 

Harlandale , Texas, Bilingual Prol an* 

Description 

tocican-American children" who were "selected a TSLS^^S ealmen- 
group had been in bilingual education classes for at least 2 years ^Je"" 
comparison group had always been in a regular English curriculum ' 



bv rhTr-^ M ! compared on average gains in grade-equivalent scores 
?U5!i "f 1 Ch8 W ° gr0Ups ^ "•*» found a « » differ on 10 

LTt^ c r aSUre) ° r / g - • Sc0rM ° f pr0grao P*"*ci?a*M and sprung and 
arithmetic computation vere ao differenn. However,' the study found chat 

scjr22S£ did ia TOcabuiar7 - rea ^- ^gua g f r a ad chac 



2-^^h^ J n f Ud , d 111 0ur v C0C4l '^nt of acceptable outcomes, cable 
2 1 and cable 2-2. dowever, because of ics use of grade-equivalencs we 
have included ic here racher ciua in chapcar 2. equivaAencs --e 
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Reasons for Re-fecg-fnn 

n™ 00 "*! 7 C ° Ua dMcri P cl °a. Olesini did not randomly select his 
*S? c0 °9* rl3on ^P 3 ' although he did match the groups on IQ 
His study is rejected, however, because of his use of grade^uivalent 

«. I LfL2f iC ^ C u° daCeralne wh " h«PP«ned even if grade-eouivaients 
are accepted. On the precesc the entering third graders scored very high- 
3.7 for the progran group end 3.5 for the comparison students. KelTher 
group can be consdiered disadvantaged (as confirmed by their IQ scores 

Srfn™!:^! * T * cd Cha ^^nt group gained 7 oonths. The historic 
performance of the treatment group was better than 1 month gain for each 
»nth of instruction (,.. their pretest score). The same w„thfcasi1or 
the comparison group. The comparison group had a bad *Iar,"ince tney £d 

ESS' £ ! 1 Chird Ch6y Salaad ^ °n*-half aonch per month of 

T J € : Cr8ata « nt S*ouP continue their historic pattern. Tnif"^^, 
strong doubts that a program effect was demonstrated. 

A second problem la found with the IQ control. If the students in the 
bilingual program «e limited in uheir English performance as a result of 
EE. S°?"? B f 11 2^? Mkiag back « rouod ' scores on a veS«I IQ test 

£H 2££ £ S) ntU3C SrtaCl7 ««»■««'*»«• their true potential 
S???r?? IQ «ore averaged 103, they must be students of very high true 
ability (as reflected in their pretest scores). 

Further, since there was no control for relative language dominance the 
!o r "co S re C s OUl f* 0n U ^ Proficient? 7 efS q ulf 

of English il rt tT t at±il ahlllCi ". *« «P«^ » differing amounts 

!flJS? ^ Chtt tow "' couid h * ve led " *«bal abilities. Their 

^S^S^^S^""" 18 co che ex " nc chac ^ L — 1 

Alice, Texas Independent School District Bilingual Education Program 
Description 

So«2si2Si^h e 2ff k±a f« r *"" B in All". Texas, were assigned co the 
Spanish-English bilingual programs on the basin of (1) test scores (2) 

.^ re ? Q C ^ a T° Val 'u aCd < 3) avai ^«- A nonraniom comparison group 

was formed by matching kindergarten language-dominance testVfor profit 

not STJ" Ce S * SinCe Che ° riSiCal doc *"««» "viewed by'SJ £d 

SLf^S! "* " 3 " ° f 3C " i3Cical significance, AIR estimated signifi- 
cance using posttest standard deviations. i 
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proj 
cane 



-***. t «u«a « r e m u ttscs, so only 5 of 16 tests of English mt- 

higher on. 7, with equivalent pretest scores la only wo nouns All hue 

-Lee T SiSnl f icaaC *** «*■ project ItSenS ocllrrtf ^ 

grades vfaere project students had an initial advantage at the stlrx of the 
school 7 ear. Project students had higher oretesc scores on 13 of"he M 

-mich the comparison group exceeded the control). 7 
Reasons tor Rejection 

*r 0 „ n ?" ^ h °! 3 r*"-* 11 aCteapC " che "eataant and comparison 

groups on the basis of class averages of students' scores on the "Oral 
Language Eligibility Test" upon entering Undergarten. Shortccinglln the 

ESfS; Jrr"' iQC?oduced hi--. Considel the kindelgarS Kettt 
scores for the 2 years covered by the report: p«c««r 

TABLE 3-10. ALICE, TEXAS, BILINGUAL PROGRAM 

SCORES ON LANGUAGE-DOMINANCE TEST 



Tear 


Spanish 
Program Comparison 


En 

Program 


glish 

Comparison 


1 
2 


57 42 
55 48 


48 
48 


49 
52 



r M li^ t !!!!™ Che w C 'u 0 f ™ V3 l00k «-°"bi7 che same in their initial 
English scores, with the comparison students being slightly ahead -he 
«eatment students show considerably better Spanifh ablli^. Cons'Jer the 
implications suggested by the data that the no groups are equivalent in 
English but on. does better in Spanish. The implication is Inat^he pro- 

?Sr£Ti2 HT^ll 17 akaU 0W£l1 ' therefore! no 

program effect needs to be postulated to account for why oro«am aareiM «»«,.« 

) "nsi«ently had higher pretest scores in' graces 1 through^ ? ana Z 

22 if S m W,t Ch8 SCh ° 01 7ear - **• is «actly attern 
that would be expected from students with greater language abilities? 



27 



143 



the authors aaver tested tL .JfJ*?? 5 pretest scores. Although 

Classroom turnover also distorts the matching procedure ever eia- 

•need a 1-yJTloI. o1 dat^Jf 2 V 8 *"* 1218 0,0 c3horts «? 8ri - 

ches. rateTare Sli^f f ? P<rC8nC ^ 34 ?erc8nC » "spectively. If 
in^he STcotorS^!? S Ch ! 78ar C ° 7841 cu "°™, the total turnover 

a* l^rcen^es^ 1°^ J™*" 88 

initial matching is unknown. 7 ° f Curaover °° «=he 

rh. L°« Ch * 0n8 ? r3d8 Wh8 " macchia « "** unaffected by turnover that is 
5 P»^*P"*» for each cohort, there 22 ao sigSli^t 

£ ilontroL 8 ^ U3h beW " n Che Pr0Sra ° P-ticipaSs aS 

in thfS-'vT S ? Ush 3C0rM £avorin * che control grcuo occur-ed 

2 I cunutltSe ^act! **+ * ^ * TSSL. 

Hnglish As A Second Language Study 

Fairfax County, 7t 77 <*u English as a Second Languar,. g roggg 
Description 

lt , ™ 8 ?airfa * Coua!: 7, Virginia, public school sys Cem operates an *nz- 
language expression, and spelling when measured (DeceaberlSlo) £Jrf is. 

?V f" 0cl: ° b « i" 7 - Students in both grates scored uSSsc J 
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'eft S e SSif t f2?» eh " aCd siKl ^« d « «o»r HSL studies, vho 
-ert the program in June 1978,. vera tested che following October. Three or- 
ate concent areas showed scores above che 50th-percenci!e level for Joch 
groups. The fourth graders performed almost at the 50th-percent"e level 

lower achievement (reading, 39th percentile, and science, 43rd percentile). 

Students in grades 4, 6, 8, and 11 categorized as speaking "aostlv a 
language other chan English" and having a -fairly good level" of £g£sh 

a SS*n!i™o^ 313 f ' cmdren scored highest in mach (45th to 

SSSS^!?-? - l0We3C in rSad±2g (13th Co 45ch P«K«eilei). Grades 
«re unixormly highest at grade 4 and lowest at grade U. 

. K „ Fairfax County study is one of the few studies in the literature 
:SLSf!J i35Ue ° f Che "tveen statistically aS educa- 

S^eLe?S^% gai f' GiVeC ^ ^ h 3an * la 3 i«s, viry smal! 
d^fere^es 5S S 1°™* "f ^"^^ significant. In many cases, these 
f «!r~;? ^ , CTUn C ° 31:7 Practical difference. Therefore, 

™^ r f^ fiCan dirft n " is 4130 bi « ecou S h *» °e educationally 
important. Following Horst et ai. (1975) and Line (1978), Fairfax defined 

US^^^^V^ 7 « " — greater haTone'tnird " 

standard deviation of the norm group. Fairfax found chac 64 of 71 comoar'- 

c'hT-SS- CrlCeri0n ° f educaci0 ^ significance. The effect of 

the rairrax program on reading scores can be seen in table 3-1 l. 

Reasons for Re lection 

in,,/ 16 ? 0 ' 1 ? 1 Che Fairfax s^dy contains much information of use to che 

55? iTSteE'Sofr? V inal ^ al7Sis che 3Cud7 addresse3 a « ue " ion 

ac^Sow wen ^! hn^ ?* 0t C ° nceni here * The ?a±rfa * looks 

natJonal nora bmnsual 3Cudencs are performing in comparison with che 



TABLE 3-11. CAT TOTAL READ DIG PERCENTILE SCORES, 
FAIRFAX COUNTY ESL PROGRAM 



Grade Precesc Posctest Gain* N 



2 22 41 



i f 9 31 12 



19 95 



114 

28 14 87 



* • 14 

5 8 li 

I 4 I 8 

8 3 li 

9 1 8 

10 1 8 

II 1 7 
12 1 7 



10 71 
73 

14 55 

8 57 

7 55 

7 67 

6 40 

6 20 



2ai - for each S^ade exceeded che educationally significant 
criterion of being larger chan one-chird s:acdard deviation of 
cne aoraicg group 
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We have discussed earlier :he subtle but iapoctanc dif-'er-nca becween 

2Sr?« « Par8 ? 1? ^° Cher * 111 Che absence of a ^reacaenc laSuSe- 

™S2 Che FairtaX 3Cud7 COBCaias =° reiav anc =o che priSipL 

concerns or che presenc scudy. H — ?a - 

3v che scandards generally used co assess che eff ecciveness of educa- 
tional programs, chese gains are very impressive. However, che ince-oreca- 
cion or che gains is not so clear-cue. Hoce chac although \£J iSgfgSs 
in percentile standing were made during just 1 year of the program, onlv 
students in grade 2 had an average percentile score above that gSn S the 

rinal level or perrormance ranged from only che 7ch co ISch percentiles. 

l QQ kJ h !^ 8 f 0re, Whil - Ch ? F3irfaX CouaC7 ? r °S raa 13 vej 7 successful when 
S h lu Mr ? " gaiC 0V " Cha 3ch001 ^ ar ' chere are grounds co cues- 

oi° IZuTl^ 4 l6Vel 0f P« rf °»«ce ac whi£ Seir Lck 
or ingUsh slcills was no longer holding chem back in school. 



Ocher Inapplicable Scudies 



ornhJS! Pf ^^=8 sections of chis chancer discussed m some decaii che 
problems v« round with a number of studies chat have regularly bee-Vced 

of Pr SrTH" f eduCaCi0n " 3u PP°"i=* ^e exLcciveness 

r K Cind Ch8 Mchodol °Sic^/logical problems encouncered in chese 
scudies co be so severe chac we cannoc accepc chem as valid evidence of 

SSS^:' 6 ": f ° r Che " aSOnS sCai * d ' * addicion, we revieweTLny 
lis" rS** u ChaC WC judged C0 be i^PPlicable. Howler, since zh£? 
ocher scudies have noc been generally, ciced as evidence for T3E? «c is coc 
necessary c 0 discuss chem ac lengch. 

we didloc a™^^"-'" Che r3a30CS for our Wl^tlon of all che scudies 
« did coc accepc. Our reasons are summarized inco eighc categories «n~ 
one oi which was sufficient co rejecc a scudy. the eig'hc cacjorles 'are' 

1. So Adjuacnenc. When scudencs are noc randomly assigned co creac- 
a«nc and control conditions, something (either matching or stacis-icS 5- 
juscmenc must be done to adjust for possible preexist dStonces a£ 
reccing learning chac could bias che resulcs of che cesct % rejected 
scudies using conrandom assignment where such adjustments were cot made. 

. 2 * gains Only. All scudencs learn someching over cine. If a scudv 

demonscraces only chac scudencs' scores wenc up oter che Sol vear ^ 
Sed wich'scoS^r eff6CC b " n de *°""aced. Gains have co be coV 

£ lt£ r / f a Pr ° P * r COnCro1 of "udeucs noc in che program 

in order Co desonscrace program ef feces. • H s 
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3. Moras. Since language-minority students do cot deveioo in English, 
their second language, the same way monolingual English speakers develop in 
English, comparing the progress of ianguage-mino ri cy student3 to notas biased 
on aono Ungual English-speaking students is not acceptable. 

4 * Criterion Tests , Criterion-referenced tests can only be used for 
our purposes if there is universal agreement about the criterion or if a 
control group was included to demonstrate a program effect. 

5. Statistics . Conclusions must be based on an appropriate statisti- 
cal analysis. If no statistical analysis was doue or if * Che analysis done 
was clearly contrary to generally accepted practice, we rejected the study. 

6« Local Criteria . Some studies are designed co address specific, 
locally relevant criteria that do not apply to our concerns. % could not 
use such results since they speak z6 different questions. 

7. GZ. As discussed in chapter 1, we do not consider grade-eauivalent 
scores co be an acceptable metric for program evaluation. 

8. Uo Detail . If a study failed to provide enough information to 
enable us to be reasonably sure none of Che above problems exists, we did 
not accept the study. 
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:.R 3-12. NBTIIOI)OM)(.'lCAI, REASONS FOR RKJRCT1NC SHIMRS* 



No Gains 
Adjustment Only No r mg 



Criterion 
Tests 



AIR , 1975e (St, John Val lny) 
AIR, 1975Q lAllce t iWTT 



J975dJ rhllodelphlo) 
_1975c (Houston) 



Ale jandro » 1979 
Alston, 1980 



iStOll 



A is ton ,""79 80 




Develo pm ent Associates, 1977 

Edwards, 1970 

Edwnrda and S m yth, 1976 
E gon and GoJd»wlth T 1901 
El**gett, 1900 



Ewnuyahyn. 19 78 



Not n .comparative oval- 
tint Ion., a study of 12 
lenrnlnR 
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(Continued) 
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TABU! 3-12. METHODOLOGICAL REASONS FOR REJECTING STUDIES* (Co.iMm.r.l) 



Fnlrfox Coun ty t 
Fort Wort hy rud 
Fc rrin, 1 979 
Cnrcln, 11*70 
Cene8ee f 1976 
Cljlnl, 1979 



Afi N ? 0ni ^ Criterion Stntin- L^J 

^iH^^^oj^ Teet^ tic, Criteria GE Detn f 1 other 




Hodlnno, 1966 
Ho nzon, 190 1 

Mul ler niHTLeo nct tT^T97^ 
M cCarth y, _n.d ' ~~ 



(Continued) 



TAHLR 3-12, METHODOLOGICAL REASONS FOR REJECTING STUDIES* 



(Continued) 



Study 



OfCcnburn * 1971 



No «ninn " Criterion 
^Adjustment Only Norms Tests 



Stntls- 
tics 



Local 
Criteria 



CE 



No 
Detail 



Other 



1973 



Good design; 

no findings available 



Uny ne and Medina. 1 9 7 2 
Trice, 1978 
Pr yor, 19 70 



Rjcl mrdj 1979 
Rlmm, 1975 
Rlmmri970 




(Continued) 



Confounded treatments 
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TABLK 3-12. METHODOLOGICAL REASONS FOR REJECTING STUDIES* (Continued) 



Vnicncla^J^L 
Valc nclo ~ 1970 



Vor lh ami Rosier, 1 979 
Wllllnk, 1968 ~ 
Winter, 1979 



Yomifl, 1980 



Z lmmer, 1976 



No 



Cnins 



All , fc . Criterion Stalls- Local nJT 

MJusUnent Only Hogg Tests tice Criteria CE De^M other 



X 



Compared 2 types ESL 



Interim report; no 
relevant results yet 



We have not listed studies tt.nt do not address our questions. 
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CHAPTER 4 
CONCLUSIONS 



Our review of the literature exploring che effectiveness of bilingual 
education has examined two basic questions pertinent to the intent of 
current Federal policy: 

1. Does transitional bilingual education lead to better 
performance in English? 

2. Does transitional bilingual education lead :o better 
performance in nonlanguage subject areas? 

We examined well over 300 documents concerned with bilingual education. Only 
28 studies were found to apply to our concerns and to meet our methodological 
criteria. These 28 methodologically sound studies included evaluations of 
pedagogical methods other chan transitional bilingu^: education, namely 
English as a second language- (ESL) and structured immersion. We present che 
results or our review an d implications for Federal policy below. 

Results 

Of the several hundred studies covered by the review, only 28* were found 
to apply to our concerns and to meet our methodological criteria. Before dis- 
cussing the studies we found to be methodologically acceptable, we should ' 
note chat we found several studies chat have previously been widely cited as 
evidence for che effectiveness of TBE .to be methodologically unacceptable 
CSkutnabb-Kangas and Toukomaa, 1976; SkuCnabb-Kangas , 1979; St. John ValW, 
1980; Jeilleux, 1977; Leyba, 1978; Trevino, 1968; Modiar.o, 1968; Egan and 
Goldsmith, 1981; Rosier and Holm, 1980; and AIR, 1975a, 1975c, 1975e). 

Table 1 summarizes the 28 studies we found to apply to our two questions 
"J," meec animal methodological criteria; by comparison, Zappert and Cruz 
round 13 methodologically acceptable studies. For each study, cable 1 
gives che auchor, che grades of school encompassed, che number of students 
in the treatment and control groups combined, che languages used by che pro- 
gram, and che resulcs che auchor(s) reporced for second-language and laach 
skills. The mosc frequenc home language w as Spanish, hue a number of other 
languages • were represenced as well. The mosc common second language was 
mglish. In chree scudies, French w as che second language. Mosc of che 
scudies were neicher longicudinal nor crue experimencs. Several scudies 
included very large numbers of students. 



* Includes che scudy by Olesini which used grade-equivalencs. 
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«"1«,1! S 2 .i,"" 6 ""'""ail? si^irlo' £ C( " ,p * rt »« *rt based 

e* al rio7T% nonlanguage subjects t^™! "ansicionai bilinzual 

e. al. (1977), 3n d Ramos ec al rto«? S ', Laobe " and Tucker M9-2, , ?, 
=each mach successfully/ u (1967) 111 showed chac i- f. 3arik 
if the wrricuS f. 7 Che Second language. Thi« % f",* " po3si °le Co 

quences from teaches i ^1*^ '/ here *^i£t£ ?~ 
. these scudleq M . H Che se cond lanffua** - "egacive conse- 

beccer chan a trf Legarreca (1979) found chac a r?F alone Was 

a THE prograni wtctaouc an ESL component ? **** ESl 

findings can be ac CribuCed Co £1™* ' indi ^ s =*an Sere ! " iult ' = he 
=o grade or becveen cescs T^t^™ 3chiev ^nc results elcn^ *' MiXed 

- - 3h 0Wing . p03 "SS^iS^ S - 

Legarret w 7 f ! ^ 3icks ' i97 8; Plan" ^.'2 < oraPariS ° n (to- 

HaftteS 197J 'c • C ° hen ' i975 >- S ™-> 1977- Lm« ^ 1968; H ^r, 19 73 

"news, 1979; Skoczylas, 1972 w„e«\ii ' and Curcis iqsn- 

l 973; Coccreli H7i. m , ' McSpadden, 1979 1980- »J 1 C' ; 

."5, c 5 s 'i/'" a - Bushe3 - " SrSlJ^ ; ™""»» 

Stern 1975 7 °°f"l*>a Wth submersion r!P e - °°° d TBE c ° h ^»e 

' l5?S ' "° 0te a ° d ? «". 1978; A. <Z£l%7y°£l 1977, 1978; 

n » McSpadden, 1980). 
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Olesiai (19 71), A. Cohen ( 1975), and .Ames and Bicks (1978) found chat 
TBE improved acquisition of math skills. However, no effect was found by 
Danoft et al. (1978), Carsrud and Curtis (1980), Moore and »arr (1978) 
McSpadden (1979, 1980) A. Cohen (1975), Covey (1973), Olesini (1971), 'sEDL 
.977 , and Ramos (1967). Skoczylas (1972), McSpadden (1980), and Stem 
reported a negative effect. 

Caution must be exercised in generalizing from table 1 .because some 
issues or aetnodological adequacy remain. For example, Covey (1973) and 
McConnell (1980a, 1980b) report success for programs including TBE. However, 
the programs also included very low staff-student ratios— 1 to 8 in the pro- 
gram studied by Covey (1980). Therefore, strong doubts exist as to whether 
the reported program effect was due to the use of bilingual instruction or 
to the small classes. 

, ^ ^^1° exanined our findings to determine which studies would have been 
included if we loosened our criteria and accepted grade-equivalents. Only 
Oiesini would then be included in our results. His results were generally 
ravorable to TBE and have been included, in table 1 and table 2. 

It is instructive to look for patterns in the findings of all these 
studies. Table 2 summarizes our findings with respect to comparing alterna- 
tive instructional approaches. .We have grouped the 28 studies according to 
the comparisons they examine. Then, we have aggregated their findings ac- 
cording to whether the study had positive, no difference, or negative re- 
sults in comparison ^.o the other approach.* For example, the first comoari- 
son in table 2 looks at the effectiveness of TBE versus submersion. For 
second-language acquisition, 10 findings favored TBE, 15 findings found no 

f rt ^ nCeS becween nE and submersion, and 5 findings were actus Uy negative 
cor TBE. ° 

The results in table 2 must be qualified. Rather than simply counting 
the number or studies with various outcomes, we must go beyond these tabula- 
tions and give more or less weight to different finding-. For example, the 
study by Ames and Bicks (1978) (which found that TBE produced better math 
results than submersion did) took place in only one school district, while 
the Danorr et al. (1978) study (which found that TBE had no effect on math) 
was designed to be nationally representative. Therefore, Danoffs findinzs 
must be given considerably more weight. Nevertheless, a clear understanding 
or our finding can only be obtained by looking at the studies in the aggre- 
gate rather than looking at the studies in isolation. Our policy imolica- 
tions are presented below. F 



* 



3ecause, as already noted, some studies had mixed results, the reader will 
notice that there are more findings than there are studies. However if 
a study administered five tests of which three had positive results 
and two negative ones, we would record only one positive and one negative 
result in our comparison tables. 



applications 



We believe the literature makes a compelling case that special programs 
ia schools can improve the achievement of language-minority children. There 
is ao evidence, however, that a specific program should be either legislated 
or preferred by the Federal Government. Indeed, more research and demonstra- 
tion projects with sound evaluation models are needed to determine which pro- 
grams are effective with which types of children in which locations. The 
rest of this summary will present our findings. 

Special Programs Can Improve Achievement la Language- 
Minority Students 

The literature we reviewed indicates that special programs designed 
to overcome language difficulties in school can improve the achievement of 
language-minority children. The studies by Pena-flughes and Solis (1980, 
19?1), PUnte (1976), Huzar (1973), Covey (1973), Kaufman (1968), and Lun 
(l. J 71) were tru*- experiments, and all showed special programs to have pos- 
itive or neutral effects. The ingenious nonexperimental design used by 
HcConnell (1980a, 1980b) also seems to have firmly established the presence 
of a positive program effect. Positive effects also were reported in the 
^experimental studies of Zirkel (1972), Ames and Bides (1968), AIR (1975b), 
Barifc and Swain (1975), Olesini (1971), Barik et al . (1979), Lambert and 
Tucker (1972), Legarreta (1979), £arsrud and Curtis (1980), Cohen (1975), 
and Malherbe (1946). Note, though, chat while special programs have been 
shown to be effective, this conclusion says nothing about the effects of any 
particular instructional approach. 



The Federal Government Should Mot Place Exclusive Reliance on 
Transitional Bilingual Education 

For more than a decade, the Federal Government has worked toward 
institutionalizing transitional bilingual 'education as virtually the only 
approved method of instruction for language-minority children, TBS has 
been emphasized in Title VII funding decisions, T3E has been implemented 
nationwide by the Office for Civil Rights' interpretation of the Lau deci- 
sion. And ia 1980, the Department of Education proposed, with, few excep- 
tions, the legal mandate of transitional bilingual education through Fed- 
eral regulations (a proposal that has been withdrawn by the current Admin- 
istration). 

When we reviewed the literature on the effectiveness of transitional 
bilingual education we did not find justification for such heavy reliance 
on this method of instruction. In order for the Federal Government to 
rely exclusively on one instructional method for meeting the needs of 
language-minority children, the following two conditions must hold; 

1. There must be a strong case that the instructional 
method is uniformly effective. 
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2. Effective instructional alternatives should not exist. If the 
desired outcomes can be reached through acre than one approach, 
the Federal Government should not constrain the options of local 
schools • 

Only 23 studies that passed our methodological test addressed the ef- 
fectiveness of TBE, and only 11 of the 25 studies looking at TEE reported 
a positive effect. Further, additional methodological problems in these 
studies impose strong limits on generalizing their results. Three studies 
suggest that the reported positive outcome could well have been due to 
other aspects of the program .rather than to TBE itself (Covey, 1973; 
McConnell, 1980a, 1980b; Plante, 1976). In additiou, a number of studies 
that used multiple-outcome measures found mixed results. Several other 
studies found a negative effe.ee for TBE when compared with submersion, 
SSL, or immersion (Dan off et al., 1977; Moore and Parr, 1978; McSpadden, 
1980; Skoczylas, 1972; Cohen, 1975; Lum, 1971; Stern, 1975; Pena-Hughes 
and Soils, 1980). Although we reviewed a limited number of immersion 
studies, each analysis of structured immersion generally found positive 
findings for that approach. Achievement in both language skill and subject 
matter knowledge was better through structured immersion than through ESL 
or T3E (3arik and Swain, 1975; Barik'et al., 1977; Lambert and Tucker, 
1972; Pena-Hughes and Solis, 1980). 

These findings <tp not add up to a very impressive case for the effec- 
tiveness of transitional bilingual education. We conclude that TBE fails 
both tests for justifying reliance on it as the exclusive method for 
instructing language-minority children. There is no firm empirical evidence 
that TBE is uniquely effective in raising language-minority students' per- 
formance in English or in nonlanguage subject areas. 

Since several States have followed the Federal lead in developing pro- 
grams for language-minority children — in some cases, even legislating T3E — 
our analysis has implications beyond the Federal level. 



Federal Policy Should Be Flexible 

For more than a decade, Federal policy (as expressed through Tide VII 
legislation, Title VII funding decisions, OCR implementation of the "Lau 
Remedies," and the August 5 Notice of Proposed Rulemaking) has emphasized 
transitional bilingual education to the virtual exclusion of alternative 
methods of instruction. We found through our analysis that this policy is 
not justified on the basis of educational effectiveness. While transitional 
bilingual education has been found to work in some settings, it has also 
been found ineffective and even harmful in other places. Furthermore, both 
of the major alternatives to TBE — structured immersion and ESL — have been 
found to work in some settings. 

The cemmonsense observation that children should be taught in a lan- 
guage they understand does not necessarily lead Co the conclusion they 
should be taught in their home language. They can be successfully taught 
in a second language if it is done right. The key to successful teaching 
in the second language seems to be to insure that the second language and 



subject matter are taught simultaneously so that subject content never 
gets ahead of language. Given the American setting, where the language- 
minority child must ultimately function in an English-speaking society, 
carefully conducted second-language instruction in all subjects may well 
be preferable to bilingual a«thods. 

We conclude that it is very hard to say what kind of program will suc- 
ceed in a particular school. Hence it seems that the only appropriate 
Federal policy is to allow schpols to develop instructional programs that 
suit the unique needs and circumstances of their students. 

There is no reason to assume a priori that the same approach that is 
applied to a rural Southwest Texas district with a large proportion of 
second-generation Hispanic children should also be applied to a district 
with a small group of Lao refugees in a Northern city. But Federax policy 
has been based on such an assumption over the years. Our review indicates 
that a fundamental change in Federal policy is needed. 

We believe this change will require recognition by the Department of 
Education that other pedagogical methods for language-minority children can be 
effective and can meet civil rights criteria. Federal funding practices 
must encompass each of the special programs designed to meet the needs of 
language -minority children so that a more realistic balance among various 
program types is achieved. 

m 

A widespread structured immersion demonstration program is especially 
needed. Until now, the immersion method has been rejected "on the basis of 
weak theoretical arguments. * * Immersion may not transfer successfully 
from Canada to the United States, but this is an empirical question that 
should be answered by direct test. As a first step, the Department should 
immediately fund an extensive evaluation of tife McAllen, Texas, prQgram, 
which has a true experimental design for comparing the effectiveness of 
structured immersion and TBE for Mexican-American students of low socio- 
economic status. 

Given the complexity of the problem, it also seems that the Federal 
Government should provide the most current information on pedagogical 
methods for language-minority children so that school districts can make 
informed choices, adapting methods to their local needs. 



Improved Bilingual Research and Program Evaluations Are Needed 

More and better research and improved program evaluations in bilingual 
education are necessary if .the needs of language-minority children are to 
be adequately met. The low quality of the methodology found throughout the 
literature is a serious problem. The major methodological problems with 
the literature include the following: 

o The absence of random assignment between treatment and control 
groups, 
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o The use of study designs chat cannot show a treatment effect in the 
absence of random assigninent , such as the norm-referenced model or 
failure co-use analysis of covariance, and 

o The failure to apply appropriate statistical tests to demonstrate 
program effects, 

- w 

ihese proolems have particularly characterized Title VII evaluations. The 
Tide VII bilingual program has begun co cake steps co improve the quality 
of local results. However-, our review has Indicated chac program evalu- 
ations are still of very poor quality; much improvement is still needed 
in this area. 

Bilingual education involves many complex, difficult issues that have 
been little (or insufficiently) studied. Federal funding for research in 
the area of bilingual education was allotted for the first time under Part 
C of Title VII in 1978, with the Elemencary and Secondary Education Act 
acervimen.es (ESEA) , The need for additional research is great, 

Unfortunately, however, when Congress established the legislation in 
i978, it limited research to examining transitional bilingual education 
specifically, rather Chan all pedagogical methods for students with limited 
English proficiency. As a result, Federal research has been skewed to 
focus on one method. Ultimately, the development of effective instructional 
programs for language-minority children will come about only through a more 
broadly based research agenda. 

Areas for redirected research should include the following: 

o A study of the divergent educational needs of language-minority 
children in the United Star.es to include the examination of how 
these children's language deficiencies differ in their home lan- 
guage and English, 

o Examination of the effectiveness of alternative instructional ap- 
proaches and how these approaches meet the needs of different 
types of language-minority children, 

o A reexamination of the theory of T3E (designed for monolingual Li 
speakers), which aay not be relevant co many of the language- 
minority students in the United States, 

o Formulation of appropriate structured immersion curricuiums, 

o Examination of the mechods of English as a second language 
(vocabulary drills versus meaningful English communication), 
and 

o Examination of bilingual education teacher qualifications and tr.e 
degree of fluency such teachers have in both languages* 
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SUMtlARY OF APPLICABLE JTUDIES 



Author 
AJR 

(Co rpus 
Chriati) 

Ames and 
flicks 



Date Grade 
!975b K-l 



1978 



BaJasu- 
brnmon- 
ian et al . 



1973 



Barik and 1975 
Swnin 



Barik 

et al, 



1977 



Cnrsrud 
and 

Curtis 



1. J 



1-9 



K-3 



K-2 



2-5 



6-5 



Design 



1 



Number 2 
of 

Students 



Languages ^ 
LI L2 



Longitudinal; 
analysis of 
covariance 

Analysis of 
covariance 



Analysis of 
covariance and 
other adjust- 
ments 



393 . Spanish English 



669 Spanish English 
and 

French 

317 Spanish English 



Longitudinal; 7,253 
analysis of 
covariance 



Longitudinal; 
analysis of 
covariance 



Longitudinal; 
analysis of 
covariance 



English French 



English French 



172 Spanish English 



L2 



Reported Results 



Math 



TBE no different from submersion 
in 1 grade; TBE better than 
submersion in 1 grade 



TBE no different from 
ESL alone 



TBE no different from 
ESL alone 



Immersion better than 
ESL 



TBE better than submer- 
sion in 1 grade; TBE 
no different from 
submersion in 1 grade 



TBE better than 
ESL alone 



Math taught in 
L2 no different 
from math taught 
in Li 

Depending on 
year and grade, 
math taught in 
L2 was worse 
than, no diff- 
erent from, or 
even better 
than math 
taught in LI 

TBE no different 
from submersion 



lb J 



(Continued) 



Author Dnte Grade 
A. Cohen 1975 K-3 



Cottrell 1971 K-l 



TWUJSJ. SUMMARY OF APPLICABLE STUDIES (Continued) 



Number 2 

i of • Lang uages^ 

Deetgnl Students LI L2 



Longitudinal 
nnalyeJ?; of 
co variance 
and other 
adjustments 



Analysis of 
covariance 



90 Spanish English 



470 Navajo English 



L2 



Jtepo rted Results 



TBE no different from 
submersion on 86 of 
100 language skills; 
submersion better than 
TBE on 11; TBP better 
than submersion on 3 



TBE no different 
from submersion 



Math 



TBE no different 
from submersion 
in 2 of 3 
grades; TBE 
better than sub- 
mersion in 1 
grade. 



Covey 



Danof f 
et al. 



ffuznr 



Kaufman 



1973 



1977, 2-6 
1978 



19 73 



1968 



2-3 



Junior 
High 



Lambert 1972 

and 
Tucker 



1-4 



Rand om 
assignment 

Analysis of 
covariance 
and other 
adjustments; 
big study 

Random assign- 
»nent; one-way 
analysis of 
covariance 

Experiment; 
longitudinal 



longitudinal; 
analysis of 
covariance 



200 Spanish English 
8,900 Several English 



160 Spanish English 



139 Spanish English 



213 English French 



(Continued) 



TBE better than sub- 
mers ion 

Submersion better than 
TBE 



TBE no different from 
subme rsion 



TBE better than submersion 
on 2 component scores of 
a standardized achievement 
test and no different on 7 
component scores i« one 
school; TBE no different 
from submersion on 9 
tests in another school. 



TBE no different 
from submersion 

TBE no different 
from submersion 



lb;. 



Math taught in 
L2 no different 
from math 
taught in LI 



16' 



Author Date Grade 



Legar- 1979 k 
reta 



Lum 



1971 



Matthews 1979 2,4, 

McConnell 1980 Pre-K 
-3 



McSpadden 1979 K-i 



McSpadden 1980 K-2 



Moore and 1978 K-2 
Fnrr 



Olesinl** 1971 J 



i u ; 



Design 



TABLE b SUMMARY OF APPLICABLE STUDIES (Continued) 

Reported Results 



Number ^ 
of 
Students 



Lang uages 3 
LI ~L2 



L2 



Ma ti» 



Analysis of 
covnriance 



Random 
assignment 



Log-linear 
model 

Longitudinal; 
subject as 
own controx 

Analysis of 
covariance 

Longitudinal ; 
analyses of 
covariance 



Analysis of 
covariance 



Ma tching 



80 



Spanish English 



55 Chinese English 

1,011 Many English 

1,020 Spanish English 

196 French English 

263 Frencli English 



130 Spanish English 



60 Spanish English 



TUB better than submersion 
or TBE no different from 
submersion, depending on 
the test; TBE with ESL better 
than TBE without ESL component 

ESL alone better than TBE on 
3 tests; ESL alone no 
different from TBE on 2 tests 

TBE/ESL no different from 
submersion 

TBE better than submersion 



TBE no different from 
submersion 

Submersion better than 
TBE in 1 of 3 grades; 
TBE no differe.it from 
submersion in 2 gtades 



Submersion better than 
TBE 



TBE better than submersion 
in 1 of 3 components 
of a standardized 
test; TBE no different 
from submersion in one 
component 



(Continued) 



TBE no different 
from submersion 

Submersion better 
than TBE in 1 
of 2 graded; 
TBE no different 
from submersion 
in 2 grades 

TBE no different 
from submersion 



TBE better than 
submersion on 1 
component of a 
standardized 
test; TBE no 
different from 
submersion on 
1 component 



Author Dnte Grade 



Pena- 
Hughes , 
and 
Soils 

Plante 



Ramos 
et al. 



SEDI, 
(Steb- 
bina) 



1900 



1976 



1967 



1977 



Skoczylns 1972 
Stern 1975 
Ztrkel 1972 



lb' J 



1-2 



1-6 



K-3 



I 

4-6 
1-3 



Random 
assignment 



TAPLE I- NUMMARY OF APPLICABLE STUDIES (Continued) 

Reported Result s 

L2 _ 

156 Spanish English 



Number ^ 
of 

Design 1 Students 



Languages 3 
LI L2 



Longitudinal ; 
experiment 



72 Spanish Engllph 



Longitudinal; 
matching 



longitudinal; 
analysis of 
covariance 
and other 
adjustments; 
5 sites 

Analysis of 
covariance 

Analysis of 
covariance 

Matching; 
analysis of 
covariance 



*** Hiligaynon English 
Pilipino 

1,060 Spanish English 



47 Spanish English 



213 Spanish English 



278 Spanish English 



(Continued) 



Immersion better than 
TBE 



TBE better than 
submersion in 1 
grade, TBE no 
different from 
submersion in 1 
grade and for both 
grade 8 combined 

TBE no different 
from immersion' 1 ,**** 



TBE no different 
from submersion' 1 



TBE no different 
from submersion 

Submersion better than 

tbf/« 

TBE better than submer- 
sion on 1 test; TBE 
no different front sub- 
mersion on h tests 



Math 



TBE no different 
from immer- 
sion^, **** 

TBE no different 
from submersion 



Submersion better 
than TBE 

Submersion better 
than TBE* 



1 '« 0 



* Treatment » 73, control not given. 

** Rejected for une of grade-equivalents only. 

*** Unable to obtain Information at present; however, the sample size was large. 

ln8trUCtl °» al — *" ".la study cannot be determined, but our beat 

3 Ll Is the language-minority child's tome language; L2 Is the child's second language. 

4 r^T" rePreSent9 COnClU8l ° n «"* the ""t^' 8 — lyfl eee chapter 2 of the full 
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TABLE 2 



SUMMARY OF FINDINGS FRCM APPLICABLE STUDIES* 
Transitional 3ilinsp al Education Versus Submersion 



TBE: f ec0nd 

Language Math 

Kwitive 10 

No Difference , f 5 " 

Nega tive 1 1 1 1 1 1 1 1 1 1 ] ] 5 3 

Transitional Billnyi al Education Versus English as a Second Language 

T3E- Second 

* ' Language Math 



Positive 

No Dif f erenc e ][[ ! 1 ! * 1 1 1 [ 

Negative lllllllllll] I 



1 1 
3 NA 
NA 



Transitional Bilingual Education Versus Immers ion* 



T3E- Second 

" Language Math 

?ositiv e 0 0 

No Difference ...... ! V 

Nega tive 1111111] 1 q 

Lmnersion Versus English as a Second Language* 



•nA^r-n^-r^* SeCOttd 

g&ffiRSI0 ^ Language Math 

Posicive 1 M 



Math scores found in immersion projects in Canada are difficult to 
compare vith scores in regular English curriculum . What can be concluded 
hovever, is that students can achieve equally well (or better) in 
math classes taught in L2 as in math classes taught in LI. 
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Appendix A 



METHODOLOGY GLOSSARY 



Age. The ease and rapidity w-_ch which children acquire a second lang- 
uage ha* been noted by parents, teachers, and linguists alike. The literature 
(Krashen, 1979) shows chat, holding cima and exposure constant— 

o Learners who begin exposure to the second language. (L2) during 
childhood generally achieve higher L2 proficiency than those 
beginning as adults. 



ter 



o Older children acquire the early stages of language skills fa: 
than younge~ children. 

o Adults proceed through the early stages of language and 
sentence development faster than children. 

Thus, adults and older children generally acquire second-language skills 
faster than younger children, but younger children will be language 
superior in the long run (Snow and Hoefnagel-Hoble, 1978). 

Some recent research relates L2 development to Piaget's theory of cog- 
nitive phases (Kessler, 1980). Children learning a second language before 
age 6 have not moved from the stage of preoperational thought to concrete 
operational thought (Piagec and Ichelder, 1969), indicating a different means 
of acquiring L2 is found in older children. Asher and Garcia (1969) found 
that immigrant children who had arrived in the United Stares between the 
ages of 1 and 6 became almost native-like English speakers. Asher and Garcia 
also presented evidence chat first starting to leam L2 between ages 7 and 10 
has detrimental effects on both languages because scund discrimination recog- 
nition as well as pronunciation begin co deceriorace ac che onsec of pubercy. 
Giles (1971) hypochesizes chat L2 instruction during ciiis age period causes 
interference becween che two languages because LI has not been fully developed 
(also see Anderson, 1978: Hughes, 1969). 

Cognitive Ability. There is an extensive literature debating whether 
bilingualism has positive or negative cognitive effects (see Darcy 1953- 
Peal and Lambert, 1962; Landry, 1974; Segalowitz, 1975; Hump.jrey, 1977- 
CoronaHo, 1979; Malherbe, 1945; Fishman, 1965; Jensen, 1963). This debate 
does not concern us except to note there are implications for the type of 
teaching methods used for bilingual children depending on the nature of the 
cognitive effects of bilingualism. However, these effects have yet to be 
well documented, so ic would be premature to comment on tnem except to note 
that the verbal skills of bilingual children are generally poorer char, 
their nonverbal skills. 

More important for our purposes is the proposition chac children differ 
in che speed wich which chey learn languages, whecher due co differences in 
flair for languages or differences in general abilicy. If chis is so, then 
more able children will acquire L2 more rapidly (see Johnson, 1953 cited i- 
Albert and Obler, 1980). Humphrey (1977) compared nonverbal IQ scores for His- 
panic children and found that the better the English language (L2) ability, 
the higher che nonverbal IQ. This finding is best interpreted as showing'" 
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rliac mastery of a second language is easier for more intelligent children. 
(See also Ardal, cited in Cummins, 1980.) A reated point is the learner's 
cognitive style (see Coronado, 1979, for a revi .w) . 

Communities. Different communities have different characteristics that 
bvtar on the teaching of language minorities. Lambert and Sidoti (1980) point 
out that in every locale languages have a status dimension. In most U.S. com- 
munities languages other than English have low prestige. However, in those 
ansa* where English is a minority language or where minority cc«^unity members 
are unafraid or losing their linguistic identity (Lambert and Tucker, 1972) 
bilingual education may w«ll yield different results. Finally, comunities 
difrer in the cumber of minority groups present. Chicago, with over 1C A lan- 
guages, faces a problem different from that of a Southwest Texas communitv 
where only EcgUsh and Spanish are typically found (for a review, see 
Skoc2ylas, 1973; Read, 1980). 

Correlation. A correlation is a measure of the extent tt, which two 
things occur together. Daylight and the presence of the sun in the sky are 
perfectly correlated— you do not have one without the other. Mcst factors - 
involved in education are not perfectly correlated. There is a high but im- 
perfect correlation between achievement and IQ. Because the correlation is 
net perfect, if * e want to understand why children differ in achievement, we 
must consider other relevant factors in addition to IQ. 

All the relationships dismissed in this report involve less than per- 
fect correlations. Readers should keep in mind that everything said about 
bilingual education is prefaced with an unstated "there is a tendency for." 

^fanidcy . The background nationality and sociocultural differences 
of the language_mlnority child can affect learning English. Several authors 
hrve suggested that learning L2 is easier the more similar L2 is to U the 
home language, LI (MacNamara, 1966). That is, it is easier for a Spanirh- 
speaking child to learn English than it is for a Vietnamese-speaking child 
because there are more linguistic similarities between English and Spanish' 
than between English and Vietnamese. Cummins (1981) also postulates that 
the sociocultural factors of a language-minority child's background affect 
language learning. In a study of 1,200 immigrants in Toronto, Cummins 
postulates sociocultural differences as the reason that children of Chinese 
background learned English better and performed better academically than 
Franco-Ontarian children. 

Gain Over the School Yeir . Several evaluations report only rhe dif- 
ference between the program students' fall and spring scores, c/en testing 
the gain for significance. This procedure is unsound. Almost all students 
show some absolute gain over time, even if they are at the same time rapidly 
falling behind the norm. 

Grade-Equivalent Scores . There are series problems with grade- 
equivalent scores, as explained in "A Prototype GvMe to Measuring Achieve- 
ment Level and Program Impact on Achievement in 3ilingual Projects" (Hcrst 
et al., 1980): 
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The aa wthorae Effect tm = < 
cnal^4 u - cclon which di - s ca^rti c^o r£ faO0U3 ° f indu - 

when they are the center of ^etenciM f * ork *" beco °e more productive 

i=8 c^SnSr cL^d^rgp ^o^th 1 ??^ faca educaCore 
English i, foreign-born or SSSS^ """J* C ° 8 ^ eak 

be randomly distributed in age Sen tnev SSf n " ^ ^ «P«" d *> 
the school i. as likely to St /^J* if ' W Khocl4 ' Therefore, 
in the tenth grade at in thf first £S Us ^^ Migrant student 
children are unlikely to mat proLL i" 17 '" 00 " 1 anoricj ■ 

English. Therefore, the typTof of!"! m ChC " UCh « rade ***** learning 
significantly with each Sc^easiS E^T^'S ° 7 £ 8011001 »" 
population is immigrant or "aSy^ dep€ndiQ * on the scuJenc 

Immigrant status has been link»H r„ „ ., 
some researchers. Canadi an reseats wperior school performance by 
the national nor. in *m^?£j? c ^^\ ia *''™ outperform 
aervacions by school personnel in rteV? \ 1* lem&i ' Anecdotal ob- 
recent Mexican immigrants o^toerfoS Jrl' !° uChwesc conclude that 

1*70; Troike, 1978; CummlJ ~ 5Sl> C ^ 0 " Mexi ^-^ricans (Carter, 
native- and foreign-boTSudren Jrftr^t \ ^ U ChaC ' alchou * h both 
migrants are more likely to ha« hfS ^ 1? 9001:17 ia SngUsh, ia- 
However, empirical evidence £ ^suptlt chela "'J 1 ' 1 " ° r 

(Kimball, 1968; Anderson ai/SoSZTlSS')! 0bSerVaCionS is liaiit ^ 

the .Ministry of Educatlo ^ and LcuS 1269?%?^ * Stud 7 ^ 

grants from culturally differ-nT 7 0Und ChaC fai ^« of immi- 

factors compounded ^ 7 e £ SaSLe ! f " "S " USed °? *><*°economic 
the educational systL and S mrrf J""^ schools, the structure of 
value system. Caraen"' a"d S^STgSj T ° ased on a *««Wt 

ibUities, dra* similar ^^^Ji.l^.SSr * lnC ° mPat - 

h-d Mtt^^ socioeconomic status 
However, Ferris (1979) founJ tbl? iJEES^SZ?" ° f 3 U8her SCaCus - 
Mexico after grade 3 did 2 .£ &£tE*i£?*™ *" W S™" d from 
born Mexican-American did. j£ or S Qra " can P^^ion as native- 
Mexican-Americans despite a *reac£ !»f r ^' noreover ' were superior to 
socioeconomic status/and' ^LTt^n 2Z» 1 «' 
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Uck of Standard Eva luation Formac . Another problem that confounds 
effectiveness comparisons of programs stems from the lack of a standardized 
evaluation format. Alkin et al. (1974) examined this problem by looking at 
SotSu ^ "?t5^ nCe in * valuaci °n "d the ise of testing data. Swain 
U978; has identified program testing as differing according to type con- 
tent, and utilization. 

Local-Cricarion-Ref er enced Designs . Communities may set up goals for 
their specific programs, such as parent involvement, curriculum development, 
and less absenteeism and then measure these goals through a criterion- 
referenced evaluation. This report is addresses a very specific Federal 
policy question dealing with student achievement aand English language ac- 
quis it. on. There may be studies chat provide perfectly adequate answers 
to locally relevant evaluacion quescions chac have no bearing on our con- 
cern here (fot example, see Offenbjrg, 1970; Young, 1980; Goodrich, 1977). 
Therefore, we can make no use of chese scudies. Cricerion-ref erenced cescs 
are further discuss* ■ in che section on cescs in chis glossary. 

Mocivacion/ Self -Concent . It has been argued by some thac many minor- 
ity languages have low prescige in che United States. The majority, there- 
fore, may view minority children as members of an inferior group, placing 
a negacive value on che children, and depressing cheir self-concepc, which 
in turn depresses school performance. De Avila and Ulibarri (1980) quescion 
chis cheory. Home language is identified wich "signif icanc ochers- who 
are very imporcanc in shaping che child's self-concepc. The degree co 
which eignificanc ochers are associaced wich reading and wricing may be 
imporcanc Co che development of self-concepc and, consequencly, liceracy 
(.Chriscian, 1976). Studies chat show greacer academic achievement and 
mocivacion by language minoricies in classes wich teachers of che s^e * 
ethnicity illustrate this point: (Modiano, 1973; Zirkel, 1972; von Malcicz, 

Lum (1971. -evievs 3everal scudies shoving thac students who accepc 
che mainstream culcure learn English more rapidly chan do chose who -cling 
to cheir own cultural group." 

Closely relaced co self-concepc is che role mocivacion plays in che 
child's school experience* Even Chough che Literature is far from conclu- 
sive as co che role played by mocivacion, ic is of cen puc forth as a plaus- 
ible explanation for discrepant findings. Ic is argued chac che child's 
mocivacion to learn L2 and to otherwise perform in school is affected by 
(see Von Maltiz, 1975; Del Buono , 1971; Skoczylas, 1973; Rand, 1980) f-hese 
factors . ' 

o Whether L2 is the dominant language of the society or a highly 
valued second language, 

o Whether the child learning 12 comes from the majority cultural 
group or from a minority group, 

o The socioeconomic status of the family, 



4 




o The linguistic pattern of the community in which the child 
lives and the degree to which the community esteems L2 and LI, 

o Parental Language use, 

o Parental attitudes toward LI and L2, 

o Subcultural differences, and 

o Parental attitudes toward school in general. 

The Novelty Effect . One interpretation of the research on educational 
improvement is that children respond to novelty. Following a change in 
the way a classroom is run, children will learn a little better. Once the 
new becomes commonplace, a change to something else again will succeed. 
Consequently, any innovative practice works when first tried, but loses 
its presumed effectiveness once it becomes the usual order of business. 

O ral or Written Language Skills . Age seems to affect acquisition of 
oral and written skills differently. C ummi ns (1978, 1980) argues there is 
little transfer (facilitation) effect from learning oral L2 to learning 
written L2. Although there is no direct evidence supporting Cummins 1 
theory, neither is there evidence refuting it. The implication is that no 
improvement in initially learning how to read (grades 1 and 2) either LI 
or L2 will result from prior instruction (preschool or grade 1) in oral 
language skills in either language (see Fishman, 1965). Conversely, 'Kramer 
C1980), Venezky (1970), and Gudschinsky (1971) argue that oral skills are 
an important precursor to learning how to read. Goodman and Goodman (1978) 
report very different oral and written skill levels in language-minority 
children. 

Parental Support . Lambert and Tucker (1972) emphasize that the St. 
Lambert program could not have been a success without the involvement of 
parents. Recognizing the importance of parental support, Title VII proj- 
ects have emphasized the role of parent advisory groups Csee also Del 
3uono, 1971). 

Post hoc Explanation s. The purpose of scientific study is to elimi- 
nate all possible explanations of an event except the one to be proved. 
If the study fails to support the chosen explanation, all the alternatives 
become viable. These after-the-fact alternative explanations are called 
post hoc explanations. Tney have very little value as scientific proof. 

The bilingual literature has many examples of post hoc explanations 
for program results. Motivation, aelf-concept, and community differences 
are usually cited. By and large, the role these factors play in bilingual 
education has not been subjected to direct tests. That they have something 
to do with the outcome of a bilingual program .s plausible, but not proven. 

Posttest-Only Design . Some studies compare only puattest 3cores of 
students in, the program and a nonrandomly selected comparison group. This 
approach is open to all the selection bias problems and the results or sucn 
studies cannot oe relied upon. 
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£E2££££Wfferenceq A - < 
cilac scudcnrTTr-rr ^porcanc character-* cm, - 

Precesc Sear* ^ 

± acor e Posctesc Score r„< 

Program _ ' * Ga -*1 



Comoarlson. " " " 

•••••••• *5 /. 
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However, de Xancer (1979) has employed a dissonance Intervention strategy 
with teachers in Texas to address attitudinal and value inconsistencies 
vnlch affect behavior, thus making teachers more open to students with English 
proficiency (also see Muller and Uonetti, 1970; Moore and Parr, 1978). 

3asically, any dimension of the school setting related to the effec- 
tiveness of schooling can affect a bilingual program. For instance, a ' 
Mjor problem in bilingual programs is the general lack of m,:erials and 
qualified teachers (NTS, 1980). Separate program elements can be P ut to- 
gether in different ways by di.'^renc schools to form a bilingual program, 
and different combinations may l«a differentially effective. 

Socioeconomic Status. Socioeconomic status (SES) has been shown to be 
at least two-thirds responsible for the relatively low achievement in school of 
Hispanics (Veltman, 1980). Moore (1978) has shown chat students of higher 
socioer.-aomic status within bilingual programs do becter than lower status 
groups. Rosenthal et al. (1981) found, in a nationally representative 
sample of elementary school students, that controlUng for socioeconomic status 
accounted for most of the low achievement of Hispanic students relative to ' 
Anglo students. De Avila (1981) concluded that controlUng for socio- 
economic status reduced the total variance accounted for among Hispanic 
students, although it did not eliminate it. In inspecting National Assess- 
ment data we also found that cognitive performance of Hispanics closely 
resembled that of blacks and SngUsh-speaking low-SES groups, while neither 
blacks nor Hispanics resembled the higher SES Anglo/white students. Thus, 
home language may not be the primary cause of low achievement for many 
children iron non-EngUsh-language backgrounds. 

Te3CS t Evaluations use nationally normed referenced tests or criterion- 
referenced tests, or both. Standardized tests permit comparisons across 
programs, but they are not completely adequate measures because many of the 
groups tested differ considerably from the norming population. There is 
also the problem of cultural biases in standardized tests. Juarez (1974) 
pointed out that students do best on tests the- take in their native tongue. 
However, Perez (1979) questioned court-ordered first-language testing in a 
study that found most of the language-minority cnUdren scored better in 
tngUsh. Lambert and Tucker (1972) found no difference in math score due 
to the language of cha cesc. 

Effecciveness scudies measure differenc skills according co the pro- 
gram goals. Oral rescs may measure faccors completely differenc frcm 
reading, wriclcg , and Uscening comprehension cescs. Similarly cescs 4 or 
cognicive skills, subjecc macter, and Unguistic proficiency measure aspects 
of a program's effectiveness but cannot be compared to one another. 

Although standardized achievement cescs have problems, a beccer alcer- 
nacive does ooc usually exisc, so we have given considerable weighc co 
sc*.cdardized achievemenc cescs. 

There are two problems wich cricerion-ref erenced cescs from our per- 
spective. Firsc, if chey are co be used in an evaluacion of a program, an 
appropriace comparison group is absolucely essencial. Second, the -selection 
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